Jul 18 01:55:04.403475 ip-10-0-130-187 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 18 01:55:04.403510 ip-10-0-130-187 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 18 01:55:04.403521 ip-10-0-130-187 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 18 01:55:04.403850 ip-10-0-130-187 systemd[1]: Failed to start Kubernetes Kubelet. Jul 18 01:55:14.564151 ip-10-0-130-187 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 18 01:55:14.564174 ip-10-0-130-187 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 0a70d45e1a294520affc2417501f6038 -- Jul 18 01:56:50.447597 ip-10-0-130-187 systemd[1]: Starting Kubernetes Kubelet... Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.834940 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840618 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840636 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840640 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840644 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840647 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840650 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:56:50.964537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840654 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:56:50.909245 ip-10-0-130-187 systemd[1]: Started Kubernetes Kubelet. Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840658 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840661 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840664 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840667 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840669 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840672 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840674 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840677 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840680 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840730 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840793 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840798 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840803 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840808 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840813 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840818 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840822 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840827 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840831 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:56:50.967220 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840836 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840840 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840845 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840850 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.840860 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841328 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841357 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841363 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841367 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841371 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841375 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841378 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841381 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841384 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841388 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841392 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841395 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841402 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841405 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841409 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:56:50.983011 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841413 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841417 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841422 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841426 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841438 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841445 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841450 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841455 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841460 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841470 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841474 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841479 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841483 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841487 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841493 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841497 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841501 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841505 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841509 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:56:50.996537 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841515 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841519 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841523 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841533 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841537 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841541 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841545 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841549 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841554 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841559 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841564 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841568 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841573 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841577 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841582 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841591 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841596 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841604 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841609 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841613 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841619 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:56:51.133460 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841623 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841627 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841631 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841635 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841640 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841643 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841653 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841657 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841662 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841666 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841673 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841678 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.841683 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842827 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842837 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842841 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842844 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842847 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842851 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842854 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:56:51.134177 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842857 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842859 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842862 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842865 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842868 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842872 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842874 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842877 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842879 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842882 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842885 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842889 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842892 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842895 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842897 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842900 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842903 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842905 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842908 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842910 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:56:51.134822 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842913 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842917 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842921 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842924 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842927 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842930 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842933 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842936 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842940 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842944 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842947 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842951 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842954 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842957 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842960 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842963 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842966 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842969 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842972 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:56:51.135666 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842976 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842979 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842982 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842985 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842988 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842991 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842993 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842996 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.842999 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843001 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843003 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843006 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843008 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843011 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843013 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843016 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843019 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843021 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843024 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843026 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:56:51.136278 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843030 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843032 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843035 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843038 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843041 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843044 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843046 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843049 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843052 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843054 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843057 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843061 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843064 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843068 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843071 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843073 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843076 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843078 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843082 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843084 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:56:51.137076 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843087 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843096 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843099 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843102 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843105 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843108 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843111 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843113 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843116 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843118 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843121 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843124 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843126 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843574 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843591 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843594 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843597 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843600 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843603 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:56:51.137701 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843606 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843608 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843611 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843614 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843617 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843619 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843622 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843624 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843627 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843629 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843632 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843637 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843641 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843644 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843647 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843650 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843652 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843655 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843658 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:56:51.138553 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843661 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843664 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843667 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843670 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843673 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843676 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843679 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843682 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843685 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843687 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843691 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843695 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843698 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843701 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843703 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843706 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843709 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843712 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843715 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:56:51.139114 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843719 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843722 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843725 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843728 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843732 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843735 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843738 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843741 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843744 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843747 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843749 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843752 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843755 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843757 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843760 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843762 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843765 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843767 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843770 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:56:51.139788 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843772 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843775 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843778 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843780 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843783 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843785 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843788 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843791 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843794 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843796 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843799 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843804 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843807 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843810 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843812 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843815 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843817 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843820 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843822 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843825 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:56:51.140311 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843829 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843832 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843835 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843837 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843840 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843843 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843845 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843848 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843850 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843853 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843856 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843859 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843862 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843864 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843867 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.843870 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.843950 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.843965 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.843972 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.843977 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.843981 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 18 01:56:51.140877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.843984 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.843989 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.843995 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.843998 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844001 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844005 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844008 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844011 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844014 2573 flags.go:64] FLAG: --cgroup-root="" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844017 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844020 2573 flags.go:64] FLAG: --client-ca-file="" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844023 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844026 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844030 2573 flags.go:64] FLAG: --cluster-domain="" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844032 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844035 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844038 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844042 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844045 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844048 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844051 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844055 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844058 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844061 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 18 01:56:51.141426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844064 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844067 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844070 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844076 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844079 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844084 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844087 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844090 2573 flags.go:64] FLAG: --enable-server="true" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844093 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844098 2573 flags.go:64] FLAG: --event-burst="100" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844102 2573 flags.go:64] FLAG: --event-qps="50" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844105 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844108 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844111 2573 flags.go:64] FLAG: --eviction-hard="" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844115 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844118 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844121 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844124 2573 flags.go:64] FLAG: --eviction-soft="" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844127 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844129 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844132 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844135 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844138 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844141 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844144 2573 flags.go:64] FLAG: --feature-gates="" Jul 18 01:56:51.142078 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844147 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844150 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844156 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844159 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844162 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844165 2573 flags.go:64] FLAG: --help="false" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844168 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844171 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844174 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844177 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844180 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844184 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844189 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844191 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844194 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844197 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844200 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844204 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844207 2573 flags.go:64] FLAG: --kube-reserved="" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844209 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844212 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844215 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844218 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844221 2573 flags.go:64] FLAG: --lock-file="" Jul 18 01:56:51.142793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844223 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844226 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844229 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844234 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844237 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844240 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844243 2573 flags.go:64] FLAG: --logging-format="text" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844245 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844249 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844251 2573 flags.go:64] FLAG: --manifest-url="" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844254 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844258 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844261 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844265 2573 flags.go:64] FLAG: --max-pods="110" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844268 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844271 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844274 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844277 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844280 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844300 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844306 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844314 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844317 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844320 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 18 01:56:51.143510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844323 2573 flags.go:64] FLAG: --pod-cidr="" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844327 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844332 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844335 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844338 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844341 2573 flags.go:64] FLAG: --port="10250" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844344 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844347 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0911337c11e9245ca" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844350 2573 flags.go:64] FLAG: --qos-reserved="" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844353 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844356 2573 flags.go:64] FLAG: --register-node="true" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844359 2573 flags.go:64] FLAG: --register-with-taints="" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844362 2573 flags.go:64] FLAG: --registry-burst="10" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844365 2573 flags.go:64] FLAG: --registry-qps="5" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844368 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844371 2573 flags.go:64] FLAG: --reserved-memory="" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844375 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844378 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844380 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844383 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844386 2573 flags.go:64] FLAG: --runonce="false" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844389 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844392 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844395 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844398 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 18 01:56:51.144560 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844400 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844404 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844407 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844410 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844415 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844418 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844421 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844424 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844428 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844431 2573 flags.go:64] FLAG: --system-cgroups="" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844434 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844440 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844443 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844445 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844449 2573 flags.go:64] FLAG: --tls-min-version="" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844452 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844455 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844458 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844462 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844464 2573 flags.go:64] FLAG: --v="2" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844468 2573 flags.go:64] FLAG: --version="false" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844472 2573 flags.go:64] FLAG: --vmodule="" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844479 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.844483 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844600 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:56:51.145263 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844605 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844608 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844610 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844613 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844615 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844619 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844621 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844625 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844630 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844635 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844639 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844644 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844648 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844651 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844654 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844659 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844662 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844665 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:56:51.279633 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844667 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844670 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844672 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844675 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844677 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844680 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844683 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844685 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844688 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844690 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844693 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844695 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844698 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844701 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844703 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844705 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844708 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844710 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844713 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844716 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:56:51.280413 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844718 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844721 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844723 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844726 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844728 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844733 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844735 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844739 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844741 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844745 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844748 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844750 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844753 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844755 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844758 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844760 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844763 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844765 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844768 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844770 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:56:51.281009 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844772 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844775 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844777 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844780 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844783 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844785 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844788 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844790 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844793 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844795 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844797 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844800 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844803 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844805 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844807 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844810 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844812 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844816 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844819 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844822 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:56:51.281605 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844825 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844829 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844832 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844834 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844837 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844839 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844842 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844844 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844847 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844850 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844852 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844855 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844857 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844860 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844862 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844865 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844867 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844870 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844872 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:56:51.282138 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.844875 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.845412 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.852210 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.852228 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.852883 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.852893 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.852976 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.852982 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.852986 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.852989 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.852992 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.852995 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.852997 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853002 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853005 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:56:51.282702 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853007 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853010 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853012 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853015 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853017 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853020 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853022 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853025 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853027 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853030 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853032 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853035 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853038 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853042 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853044 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853047 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853049 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853052 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853054 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853057 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:56:51.283119 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853061 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853065 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853069 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853072 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853076 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853079 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853082 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853084 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853087 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853090 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853093 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853096 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853098 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853101 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853103 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853106 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853108 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853111 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853113 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:56:51.283725 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853118 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853121 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853124 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853126 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853129 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853132 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853135 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853138 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853140 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853143 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853145 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853148 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853151 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853153 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853156 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853160 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853163 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853166 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853169 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853171 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:56:51.284223 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853174 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853177 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853179 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853182 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853185 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853188 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853191 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853194 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853198 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853201 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853204 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853207 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853211 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853213 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853216 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853218 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853221 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853225 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853228 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:56:51.284805 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853230 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853233 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853235 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853238 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853240 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853243 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853246 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853248 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853251 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853253 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853256 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853258 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.853264 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853391 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853396 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:56:51.285367 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853399 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853402 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853405 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853409 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853413 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853416 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853419 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853422 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853424 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853427 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853430 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853435 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853439 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853442 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853445 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853450 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853453 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853456 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853458 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:56:51.285868 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853461 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853463 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853466 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853469 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853471 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853474 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853477 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853479 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853482 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853485 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853487 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853490 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853492 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853495 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853499 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853502 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853504 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853507 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853510 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:56:51.286403 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853513 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853515 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853518 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853520 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853523 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853526 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853529 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853532 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853535 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853538 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853541 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853544 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853546 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853549 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853551 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853554 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853556 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853559 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853562 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853564 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:56:51.286927 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853567 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853569 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853572 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853575 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853578 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853580 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853583 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853585 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853588 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853591 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853610 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853614 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853618 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853621 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853624 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853628 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853630 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853634 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853637 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853639 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:56:51.287479 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853642 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853646 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853649 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853651 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853654 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853657 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853659 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853662 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853664 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853667 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853670 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853673 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853676 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853678 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853681 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853684 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853686 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853689 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:56:51.288046 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:50.853691 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.853696 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.853825 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.855899 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.857310 2573 server.go:1019] "Starting client certificate rotation" Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.857419 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.857997 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.859819 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.861317 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.874665 2573 log.go:25] "Validated CRI v1 runtime API" Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.878760 2573 log.go:25] "Validated CRI v1 image API" Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.880206 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 18 01:56:51.288539 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.883353 2573 fs.go:135] Filesystem UUIDs: map[114b3f5c-8118-45d8-8f3e-a24dc0cf3184:/dev/nvme0n1p4 59e1010c-6c47-4b1a-a1a5-3bfb5cd69a6f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 18 01:56:51.288920 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.883371 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 18 01:56:51.288986 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.888365 2573 manager.go:216] Machine: {Timestamp:2026-07-18 01:56:50.887164452 +0000 UTC m=+0.321185805 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097068 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2463602904251f0504f15bd4031995 SystemUUID:ec246360-2904-251f-0504-f15bd4031995 BootID:0a70d45e-1a29-4520-affc-2417501f6038 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9b:90:30:2d:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9b:90:30:2d:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:ab:5d:b3:3f:05 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 18 01:56:51.288986 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.888483 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 18 01:56:51.288986 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.888576 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 18 01:56:51.288986 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.890722 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 01:56:51.288986 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.891375 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 18 01:56:51.288986 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.891414 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-187.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 18 01:56:51.288986 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.892054 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.892065 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.892072 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.892678 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.893844 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.896994 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.897115 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.897143 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.897164 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.897181 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.897190 2573 kubelet.go:398] "Adding apiserver pod source" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.897200 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.897595 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.897613 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.900097 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.902264 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.902305 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904084 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904098 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904104 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904109 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904118 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904123 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904131 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904137 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904145 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904154 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904165 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904171 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904863 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.904872 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.906397 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-187.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:50.906521 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:50.906599 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-187.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.908349 2573 server.go:1268] "Started kubelet" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.908417 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.908494 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.908595 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.910317 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2z4z8" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.910419 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 18 01:56:51.289268 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.913327 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:50.915028 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.915258 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2z4z8" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.919298 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.919313 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:50.919417 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-187.ec2.internal\" not found" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.919856 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.919957 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.919971 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.920050 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.920057 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.920448 2573 factory.go:55] Registering systemd factory Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.920465 2573 factory.go:223] Registration of the systemd container factory successfully Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.920680 2573 factory.go:153] Registering CRI-O factory Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.920692 2573 factory.go:223] Registration of the crio container factory successfully Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.920736 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.920758 2573 factory.go:103] Registering Raw factory Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.920770 2573 manager.go:1196] Started watching for new ooms in manager Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.921258 2573 manager.go:349] Starting recovery of all containers Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.923021 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:50.927152 2573 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-187.ec2.internal\" not found" node="ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.931925 2573 manager.go:354] Recovery completed Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.937001 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.939032 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.939064 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.939076 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.939672 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.939679 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.939689 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.939696 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.940436 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.940462 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.940471 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.940785 2573 policy_none.go:49] "None policy: Start" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.940798 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.940808 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:50.941835 2573 policy_none.go:47] "Start" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.014937 2573 manager.go:337] "Starting Device Plugin manager" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.015030 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.015045 2573 server.go:87] "Starting device plugin registration server" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.015350 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.015362 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.015553 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.015629 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 18 01:56:51.290907 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.015638 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.016366 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.016445 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-187.ec2.internal\" not found" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.069233 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.070473 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.070510 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.070530 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.070565 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.073275 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.115531 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.116547 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.116577 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.116592 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.116626 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.125897 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.125917 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-187.ec2.internal\": node \"ip-10-0-130-187.ec2.internal\" not found" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.139259 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-187.ec2.internal\" not found" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.171344 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal"] Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.171431 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.174261 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.174304 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.174316 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.175714 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.175826 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.175860 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.176496 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.176525 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.176538 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.176987 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.177017 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.177033 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.178518 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.178547 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.179196 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:56:51.431683 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.179220 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.179232 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.194879 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-187.ec2.internal\" not found" node="ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.199547 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-187.ec2.internal\" not found" node="ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.221568 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/db31749ad92473fcf6d05f4eabaac6ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal\" (UID: \"db31749ad92473fcf6d05f4eabaac6ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.221605 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db31749ad92473fcf6d05f4eabaac6ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal\" (UID: \"db31749ad92473fcf6d05f4eabaac6ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.221632 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e85115e7389ad24d665892c86f4108f1-config\") pod \"kube-apiserver-proxy-ip-10-0-130-187.ec2.internal\" (UID: \"e85115e7389ad24d665892c86f4108f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.240348 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-187.ec2.internal\" not found" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.322177 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/db31749ad92473fcf6d05f4eabaac6ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal\" (UID: \"db31749ad92473fcf6d05f4eabaac6ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.322207 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db31749ad92473fcf6d05f4eabaac6ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal\" (UID: \"db31749ad92473fcf6d05f4eabaac6ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.322225 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e85115e7389ad24d665892c86f4108f1-config\") pod \"kube-apiserver-proxy-ip-10-0-130-187.ec2.internal\" (UID: \"e85115e7389ad24d665892c86f4108f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.322300 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/db31749ad92473fcf6d05f4eabaac6ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal\" (UID: \"db31749ad92473fcf6d05f4eabaac6ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.322356 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e85115e7389ad24d665892c86f4108f1-config\") pod \"kube-apiserver-proxy-ip-10-0-130-187.ec2.internal\" (UID: \"e85115e7389ad24d665892c86f4108f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.322382 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db31749ad92473fcf6d05f4eabaac6ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal\" (UID: \"db31749ad92473fcf6d05f4eabaac6ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.438707 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.341274 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-187.ec2.internal\" not found" Jul 18 01:56:51.442463 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.442166 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-187.ec2.internal\" not found" Jul 18 01:56:51.503561 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.498356 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.503561 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.502844 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.659996 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.542789 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-187.ec2.internal\" not found" Jul 18 01:56:51.659996 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.643305 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-187.ec2.internal\" not found" Jul 18 01:56:51.827611 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.743796 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-187.ec2.internal\" not found" Jul 18 01:56:51.827611 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.791514 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:56:51.827611 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.819494 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.830182 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.831928 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.840801 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.856957 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.857103 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.857130 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.857139 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.898264 2573 apiserver.go:52] "Watching apiserver" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.914556 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.915449 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal","openshift-multus/multus-additional-cni-plugins-rg7tx","openshift-multus/multus-c6q2t","openshift-network-diagnostics/network-check-target-m5dx8","kube-system/global-pull-secret-syncer-ljk5t","kube-system/konnectivity-agent-9fbkh","openshift-cluster-node-tuning-operator/tuned-jxvhp","openshift-dns/node-resolver-fpf6q","openshift-multus/network-metrics-daemon-zjvkm","openshift-network-operator/iptables-alerter-4h6fl","openshift-ovn-kubernetes/ovnkube-node-dgqnb","kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v","openshift-image-registry/node-ca-m2vjh"] Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.916790 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.918702 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.918873 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.918879 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.918953 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5nr9r\"" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.919244 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 01:51:52 +0000 UTC" deadline="2027-12-13 01:33:11.94893135 +0000 UTC" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.919272 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12311h36m20.029664699s" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.919487 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.919856 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.920678 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.920731 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hqh5h\"" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.921481 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.921591 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:56:51.937472 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.921799 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.922170 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.922529 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.922572 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.922630 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m8wvx\"" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.923135 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.923140 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.924326 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.924716 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.925777 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-run-k8s-cni-cncf-io\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.925803 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-daemon-config\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.925820 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-sysctl-conf\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.925826 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.925835 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-systemd\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.925857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-var-lib-kubelet\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.925881 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-device-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.925933 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-etc-selinux\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:51.938320 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.925960 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-sys-fs\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926010 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-socket-dir-parent\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926036 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-var-lib-cni-multus\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926054 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-var-lib-kubelet\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926079 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkt4b\" (UniqueName: \"kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b\") pod \"network-check-target-m5dx8\" (UID: \"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5\") " pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926098 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-var-lib-cni-bin\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926113 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-run-multus-certs\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926133 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-etc-kubernetes\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926161 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-sysctl-d\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926179 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-lib-modules\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926199 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-system-cni-dir\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926217 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-os-release\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926232 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926246 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-socket-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:51.938800 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926263 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-registration-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926299 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-modprobe-d\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926341 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdks7\" (UniqueName: \"kubernetes.io/projected/ce91eb83-04ec-4406-b92d-42beec0e867e-kube-api-access-qdks7\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926359 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-run-netns\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926381 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqrn9\" (UniqueName: \"kubernetes.io/projected/83f2049b-7030-439f-843a-bda5238b11dc-kube-api-access-wqrn9\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926401 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-sysconfig\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-run\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926430 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-cni-dir\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926443 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-cni-binary-copy\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926472 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-hostroot\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926487 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-conf-dir\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926500 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfwkq\" (UniqueName: \"kubernetes.io/projected/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-kube-api-access-rfwkq\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926514 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-sys\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-host\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.939674 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926556 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/83f2049b-7030-439f-843a-bda5238b11dc-etc-tuned\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926572 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-cnibin\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926603 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-kubernetes\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.926618 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83f2049b-7030-439f-843a-bda5238b11dc-tmp\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.927022 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:51.927100 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.927771 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.927829 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-h7jnw\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.928054 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.928120 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.929512 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.929991 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.930183 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jhg59\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.930269 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.930536 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.930700 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.931211 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.931471 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.931686 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2l9g5\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.932220 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.932771 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.932810 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.932821 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.932823 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.933092 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4jb2b\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.933558 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.934150 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t7grl\"" Jul 18 01:56:51.940085 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.934380 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 18 01:56:51.940903 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.934433 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 18 01:56:51.940903 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.934660 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 18 01:56:51.940903 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.935046 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 18 01:56:51.940903 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.935112 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 18 01:56:51.940903 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.935488 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 18 01:56:51.940903 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.936057 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 18 01:56:51.940903 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.936066 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l9q98\"" Jul 18 01:56:51.940903 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.936064 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.953529 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qrwvs" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:51.958572 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qrwvs" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.021775 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026797 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-cnibin\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026821 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83f2049b-7030-439f-843a-bda5238b11dc-tmp\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026840 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-slash\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026860 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-cni-netd\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026887 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-sysctl-conf\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026902 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-systemd\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026917 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/288254b4-f459-4f1f-bb45-ae9180fbefa8-cni-binary-copy\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026936 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/087b9933-3cbe-42c2-bbc1-78326b899fe2-host-slash\") pod \"iptables-alerter-4h6fl\" (UID: \"087b9933-3cbe-42c2-bbc1-78326b899fe2\") " pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026957 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-etc-selinux\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026978 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-sys-fs\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026978 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-cnibin\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026989 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-systemd\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.039047 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027000 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkt4b\" (UniqueName: \"kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b\") pod \"network-check-target-m5dx8\" (UID: \"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5\") " pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.026992 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-sysctl-conf\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027023 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-ovn-node-metrics-cert\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027037 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-sys-fs\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027042 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f6ca5e1-acf6-4e50-a04a-a0d039d44da9-host\") pod \"node-ca-m2vjh\" (UID: \"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9\") " pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027078 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tclkf\" (UniqueName: \"kubernetes.io/projected/3f6ca5e1-acf6-4e50-a04a-a0d039d44da9-kube-api-access-tclkf\") pod \"node-ca-m2vjh\" (UID: \"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9\") " pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027093 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-etc-selinux\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027101 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-os-release\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027143 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027175 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-etc-kubernetes\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027201 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-kubelet-config\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027227 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1827c91-f990-469b-b0df-d612d539262c-agent-certs\") pod \"konnectivity-agent-9fbkh\" (UID: \"f1827c91-f990-469b-b0df-d612d539262c\") " pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027246 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-etc-kubernetes\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027250 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-system-cni-dir\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.040300 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027306 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-system-cni-dir\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027311 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-os-release\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027344 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-registration-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-modprobe-d\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-dbus\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027428 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-registration-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-run-openvswitch\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027461 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kkzv\" (UniqueName: \"kubernetes.io/projected/4c6efe1e-41a0-429b-a45e-678499ab8f80-kube-api-access-8kkzv\") pod \"node-resolver-fpf6q\" (UID: \"4c6efe1e-41a0-429b-a45e-678499ab8f80\") " pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027464 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-modprobe-d\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027471 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-os-release\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027500 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-run-netns\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027519 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqrn9\" (UniqueName: \"kubernetes.io/projected/83f2049b-7030-439f-843a-bda5238b11dc-kube-api-access-wqrn9\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-ovnkube-config\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027548 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-env-overrides\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.040976 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027557 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-run-netns\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027563 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-system-cni-dir\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027608 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/288254b4-f459-4f1f-bb45-ae9180fbefa8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027631 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1827c91-f990-469b-b0df-d612d539262c-konnectivity-ca\") pod \"konnectivity-agent-9fbkh\" (UID: \"f1827c91-f990-469b-b0df-d612d539262c\") " pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027687 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-cni-dir\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027732 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-hostroot\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027797 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfwkq\" (UniqueName: \"kubernetes.io/projected/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-kube-api-access-rfwkq\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027821 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-sys\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027843 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-host\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027862 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-hostroot\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027897 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-sys\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027908 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9jnm\" (UniqueName: \"kubernetes.io/projected/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-kube-api-access-p9jnm\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027883 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-cni-dir\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027927 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-host\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027928 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c6efe1e-41a0-429b-a45e-678499ab8f80-tmp-dir\") pod \"node-resolver-fpf6q\" (UID: \"4c6efe1e-41a0-429b-a45e-678499ab8f80\") " pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:52.041651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027945 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027966 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-kubernetes\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.027993 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-systemd-units\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028017 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-run-systemd\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028036 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-kubernetes\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028056 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-ovnkube-script-lib\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028079 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-677fw\" (UniqueName: \"kubernetes.io/projected/087b9933-3cbe-42c2-bbc1-78326b899fe2-kube-api-access-677fw\") pod \"iptables-alerter-4h6fl\" (UID: \"087b9933-3cbe-42c2-bbc1-78326b899fe2\") " pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028117 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-run-k8s-cni-cncf-io\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028149 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-daemon-config\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028154 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-run-k8s-cni-cncf-io\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028176 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-var-lib-kubelet\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028202 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v64w\" (UniqueName: \"kubernetes.io/projected/2992267b-15c9-4fa5-be5d-d783f560f9a6-kube-api-access-7v64w\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028220 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-kubelet\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028241 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-log-socket\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.042208 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028265 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-var-lib-kubelet\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028299 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/087b9933-3cbe-42c2-bbc1-78326b899fe2-iptables-alerter-script\") pod \"iptables-alerter-4h6fl\" (UID: \"087b9933-3cbe-42c2-bbc1-78326b899fe2\") " pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028329 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-etc-openvswitch\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-device-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028387 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-socket-dir-parent\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-var-lib-cni-multus\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028437 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-var-lib-kubelet\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028445 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-socket-dir-parent\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028458 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-sysctl-d\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028481 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-var-lib-cni-multus\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028490 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/288254b4-f459-4f1f-bb45-ae9180fbefa8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028443 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-device-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028499 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-var-lib-kubelet\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028519 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-var-lib-cni-bin\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.042871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028544 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-run-multus-certs\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028552 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-sysctl-d\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028562 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-lib-modules\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028586 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028599 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-run-multus-certs\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028608 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028608 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-host-var-lib-cni-bin\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028636 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-node-log\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028660 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-cni-bin\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028663 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-daemon-config\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028672 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-lib-modules\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028710 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028730 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-socket-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.043372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028746 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdks7\" (UniqueName: \"kubernetes.io/projected/ce91eb83-04ec-4406-b92d-42beec0e867e-kube-api-access-qdks7\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028770 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-run-netns\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028791 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028797 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-run-ovn\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028829 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f6ca5e1-acf6-4e50-a04a-a0d039d44da9-serviceca\") pod \"node-ca-m2vjh\" (UID: \"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9\") " pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028848 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-cnibin\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028862 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpv7c\" (UniqueName: \"kubernetes.io/projected/288254b4-f459-4f1f-bb45-ae9180fbefa8-kube-api-access-lpv7c\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028880 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-sysconfig\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028899 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-run\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028893 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce91eb83-04ec-4406-b92d-42beec0e867e-socket-dir\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028936 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-run\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028937 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/83f2049b-7030-439f-843a-bda5238b11dc-etc-sysconfig\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028955 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028979 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4c6efe1e-41a0-429b-a45e-678499ab8f80-hosts-file\") pod \"node-resolver-fpf6q\" (UID: \"4c6efe1e-41a0-429b-a45e-678499ab8f80\") " pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:52.043951 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.028997 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-cni-binary-copy\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.029011 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-conf-dir\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.029032 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/83f2049b-7030-439f-843a-bda5238b11dc-etc-tuned\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.029046 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-var-lib-openvswitch\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.029099 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-multus-conf-dir\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.029489 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-cni-binary-copy\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.031058 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83f2049b-7030-439f-843a-bda5238b11dc-tmp\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.031070 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/83f2049b-7030-439f-843a-bda5238b11dc-etc-tuned\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:52.035481 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:52.035500 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:52.035511 2573 projected.go:196] Error preparing data for projected volume kube-api-access-xkt4b for pod openshift-network-diagnostics/network-check-target-m5dx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:52.035571 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b podName:009eb0e0-baa3-4a3a-94e3-2ce563f67bd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:52.535553139 +0000 UTC m=+1.969574493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xkt4b" (UniqueName: "kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b") pod "network-check-target-m5dx8" (UID: "009eb0e0-baa3-4a3a-94e3-2ce563f67bd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.037241 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdks7\" (UniqueName: \"kubernetes.io/projected/ce91eb83-04ec-4406-b92d-42beec0e867e-kube-api-access-qdks7\") pod \"aws-ebs-csi-driver-node-glx6v\" (UID: \"ce91eb83-04ec-4406-b92d-42beec0e867e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.037272 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfwkq\" (UniqueName: \"kubernetes.io/projected/1ff6170e-2dcc-4e43-b1d2-cac6fccc2339-kube-api-access-rfwkq\") pod \"multus-c6q2t\" (UID: \"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339\") " pod="openshift-multus/multus-c6q2t" Jul 18 01:56:52.044510 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:52.037797 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqrn9\" (UniqueName: \"kubernetes.io/projected/83f2049b-7030-439f-843a-bda5238b11dc-kube-api-access-wqrn9\") pod \"tuned-jxvhp\" (UID: \"83f2049b-7030-439f-843a-bda5238b11dc\") " pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:54.032319 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:53.943148 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576193 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576296 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c6q2t" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576314 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-kubelet-config\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576342 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1827c91-f990-469b-b0df-d612d539262c-agent-certs\") pod \"konnectivity-agent-9fbkh\" (UID: \"f1827c91-f990-469b-b0df-d612d539262c\") " pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576368 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-dbus\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576389 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-run-openvswitch\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576411 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kkzv\" (UniqueName: \"kubernetes.io/projected/4c6efe1e-41a0-429b-a45e-678499ab8f80-kube-api-access-8kkzv\") pod \"node-resolver-fpf6q\" (UID: \"4c6efe1e-41a0-429b-a45e-678499ab8f80\") " pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576457 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-kubelet-config\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576480 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-dbus\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576511 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-run-openvswitch\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576515 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-ovnkube-config\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576547 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-env-overrides\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576578 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-system-cni-dir\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576604 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/288254b4-f459-4f1f-bb45-ae9180fbefa8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.659663 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576628 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1827c91-f990-469b-b0df-d612d539262c-konnectivity-ca\") pod \"konnectivity-agent-9fbkh\" (UID: \"f1827c91-f990-469b-b0df-d612d539262c\") " pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9jnm\" (UniqueName: \"kubernetes.io/projected/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-kube-api-access-p9jnm\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576700 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c6efe1e-41a0-429b-a45e-678499ab8f80-tmp-dir\") pod \"node-resolver-fpf6q\" (UID: \"4c6efe1e-41a0-429b-a45e-678499ab8f80\") " pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576724 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-systemd-units\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576777 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-run-systemd\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576805 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-ovnkube-script-lib\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576839 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-677fw\" (UniqueName: \"kubernetes.io/projected/087b9933-3cbe-42c2-bbc1-78326b899fe2-kube-api-access-677fw\") pod \"iptables-alerter-4h6fl\" (UID: \"087b9933-3cbe-42c2-bbc1-78326b899fe2\") " pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576864 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v64w\" (UniqueName: \"kubernetes.io/projected/2992267b-15c9-4fa5-be5d-d783f560f9a6-kube-api-access-7v64w\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576886 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-kubelet\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-log-socket\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576929 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/087b9933-3cbe-42c2-bbc1-78326b899fe2-iptables-alerter-script\") pod \"iptables-alerter-4h6fl\" (UID: \"087b9933-3cbe-42c2-bbc1-78326b899fe2\") " pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.576960 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-etc-openvswitch\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577006 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/288254b4-f459-4f1f-bb45-ae9180fbefa8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577038 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:54.660422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577094 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-node-log\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577116 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-cni-bin\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577136 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-env-overrides\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577155 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-ovnkube-config\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577140 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577182 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-system-cni-dir\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577205 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577219 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-run-netns\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577251 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-run-netns\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577251 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-run-ovn\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577264 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1827c91-f990-469b-b0df-d612d539262c-konnectivity-ca\") pod \"konnectivity-agent-9fbkh\" (UID: \"f1827c91-f990-469b-b0df-d612d539262c\") " pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577281 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f6ca5e1-acf6-4e50-a04a-a0d039d44da9-serviceca\") pod \"node-ca-m2vjh\" (UID: \"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9\") " pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577282 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-run-ovn\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662076 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577360 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-cnibin\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577378 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/288254b4-f459-4f1f-bb45-ae9180fbefa8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577391 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpv7c\" (UniqueName: \"kubernetes.io/projected/288254b4-f459-4f1f-bb45-ae9180fbefa8-kube-api-access-lpv7c\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577421 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577493 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-systemd-units\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577526 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-run-systemd\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577558 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577589 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-cnibin\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577625 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-node-log\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4c6efe1e-41a0-429b-a45e-678499ab8f80-hosts-file\") pod \"node-resolver-fpf6q\" (UID: \"4c6efe1e-41a0-429b-a45e-678499ab8f80\") " pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577702 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-ovnkube-script-lib\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:54.577711 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-var-lib-openvswitch\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:54.577768 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs podName:2992267b-15c9-4fa5-be5d-d783f560f9a6 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:55.077750275 +0000 UTC m=+2.866086863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs") pod "network-metrics-daemon-zjvkm" (UID: "2992267b-15c9-4fa5-be5d-d783f560f9a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:56:54.662877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577780 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-var-lib-openvswitch\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577808 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-cni-bin\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:54.577822 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577836 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-log-socket\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577848 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c6efe1e-41a0-429b-a45e-678499ab8f80-tmp-dir\") pod \"node-resolver-fpf6q\" (UID: \"4c6efe1e-41a0-429b-a45e-678499ab8f80\") " pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577854 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-kubelet\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:54.577894 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret podName:5c41bb40-6562-4a53-8641-8cb1b0a2c4f3 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:55.077877134 +0000 UTC m=+2.866213696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret") pod "global-pull-secret-syncer-ljk5t" (UID: "5c41bb40-6562-4a53-8641-8cb1b0a2c4f3") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577892 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-etc-openvswitch\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577929 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-slash\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577950 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-cni-netd\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.577990 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/288254b4-f459-4f1f-bb45-ae9180fbefa8-cni-binary-copy\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578011 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/087b9933-3cbe-42c2-bbc1-78326b899fe2-host-slash\") pod \"iptables-alerter-4h6fl\" (UID: \"087b9933-3cbe-42c2-bbc1-78326b899fe2\") " pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578019 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4c6efe1e-41a0-429b-a45e-678499ab8f80-hosts-file\") pod \"node-resolver-fpf6q\" (UID: \"4c6efe1e-41a0-429b-a45e-678499ab8f80\") " pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578039 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/087b9933-3cbe-42c2-bbc1-78326b899fe2-iptables-alerter-script\") pod \"iptables-alerter-4h6fl\" (UID: \"087b9933-3cbe-42c2-bbc1-78326b899fe2\") " pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:54.578138 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:56:54.664087 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:54.578157 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:54.578172 2573 projected.go:196] Error preparing data for projected volume kube-api-access-xkt4b for pod openshift-network-diagnostics/network-check-target-m5dx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:54.578217 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b podName:009eb0e0-baa3-4a3a-94e3-2ce563f67bd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:55.578202427 +0000 UTC m=+3.366539001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xkt4b" (UniqueName: "kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b") pod "network-check-target-m5dx8" (UID: "009eb0e0-baa3-4a3a-94e3-2ce563f67bd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578048 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkt4b\" (UniqueName: \"kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b\") pod \"network-check-target-m5dx8\" (UID: \"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5\") " pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578258 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-ovn-node-metrics-cert\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578284 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f6ca5e1-acf6-4e50-a04a-a0d039d44da9-host\") pod \"node-ca-m2vjh\" (UID: \"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9\") " pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578312 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tclkf\" (UniqueName: \"kubernetes.io/projected/3f6ca5e1-acf6-4e50-a04a-a0d039d44da9-kube-api-access-tclkf\") pod \"node-ca-m2vjh\" (UID: \"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9\") " pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578342 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-os-release\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578378 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/288254b4-f459-4f1f-bb45-ae9180fbefa8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578435 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-slash\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578459 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f6ca5e1-acf6-4e50-a04a-a0d039d44da9-serviceca\") pod \"node-ca-m2vjh\" (UID: \"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9\") " pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578477 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-host-cni-netd\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578481 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/288254b4-f459-4f1f-bb45-ae9180fbefa8-cni-binary-copy\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.664866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578510 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/087b9933-3cbe-42c2-bbc1-78326b899fe2-host-slash\") pod \"iptables-alerter-4h6fl\" (UID: \"087b9933-3cbe-42c2-bbc1-78326b899fe2\") " pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578524 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578376 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578620 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f6ca5e1-acf6-4e50-a04a-a0d039d44da9-host\") pod \"node-ca-m2vjh\" (UID: \"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9\") " pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.578620 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/288254b4-f459-4f1f-bb45-ae9180fbefa8-os-release\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.579268 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1827c91-f990-469b-b0df-d612d539262c-agent-certs\") pod \"konnectivity-agent-9fbkh\" (UID: \"f1827c91-f990-469b-b0df-d612d539262c\") " pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.581273 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-ovn-node-metrics-cert\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.581784 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.587351 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpv7c\" (UniqueName: \"kubernetes.io/projected/288254b4-f459-4f1f-bb45-ae9180fbefa8-kube-api-access-lpv7c\") pod \"multus-additional-cni-plugins-rg7tx\" (UID: \"288254b4-f459-4f1f-bb45-ae9180fbefa8\") " pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.587749 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tclkf\" (UniqueName: \"kubernetes.io/projected/3f6ca5e1-acf6-4e50-a04a-a0d039d44da9-kube-api-access-tclkf\") pod \"node-ca-m2vjh\" (UID: \"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9\") " pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.587875 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-677fw\" (UniqueName: \"kubernetes.io/projected/087b9933-3cbe-42c2-bbc1-78326b899fe2-kube-api-access-677fw\") pod \"iptables-alerter-4h6fl\" (UID: \"087b9933-3cbe-42c2-bbc1-78326b899fe2\") " pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.588848 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v64w\" (UniqueName: \"kubernetes.io/projected/2992267b-15c9-4fa5-be5d-d783f560f9a6-kube-api-access-7v64w\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.589098 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kkzv\" (UniqueName: \"kubernetes.io/projected/4c6efe1e-41a0-429b-a45e-678499ab8f80-kube-api-access-8kkzv\") pod \"node-resolver-fpf6q\" (UID: \"4c6efe1e-41a0-429b-a45e-678499ab8f80\") " pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.589146 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9jnm\" (UniqueName: \"kubernetes.io/projected/2e4eb8b4-5a82-4673-a734-1a2d3842c1a8-kube-api-access-p9jnm\") pod \"ovnkube-node-dgqnb\" (UID: \"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.596591 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m2vjh" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.606529 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:51:53 +0000 UTC" deadline="2028-02-02 03:19:39.502085456 +0000 UTC" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.606573 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13537h22m44.895516041s" Jul 18 01:56:54.666486 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.611685 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:56:54.738121 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.738099 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:56:54.750013 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.749990 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" Jul 18 01:56:54.768425 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.768401 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4h6fl" Jul 18 01:56:54.780042 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.780009 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:56:54.788058 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.788037 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fpf6q" Jul 18 01:56:54.796090 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:54.796061 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85115e7389ad24d665892c86f4108f1.slice/crio-0df06cebe796f2f928edb94f69c218b101b7fb578e6eb92468ef22b7db4cb29c WatchSource:0}: Error finding container 0df06cebe796f2f928edb94f69c218b101b7fb578e6eb92468ef22b7db4cb29c: Status 404 returned error can't find the container with id 0df06cebe796f2f928edb94f69c218b101b7fb578e6eb92468ef22b7db4cb29c Jul 18 01:56:54.796961 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:54.796937 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod087b9933_3cbe_42c2_bbc1_78326b899fe2.slice/crio-a7fa0ba492b55b2faf5dfcbb283b3cca9861e70d61a28b279abcec1b485f2f39 WatchSource:0}: Error finding container a7fa0ba492b55b2faf5dfcbb283b3cca9861e70d61a28b279abcec1b485f2f39: Status 404 returned error can't find the container with id a7fa0ba492b55b2faf5dfcbb283b3cca9861e70d61a28b279abcec1b485f2f39 Jul 18 01:56:54.798226 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:54.798209 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e4eb8b4_5a82_4673_a734_1a2d3842c1a8.slice/crio-c5a4cdfdf01f7b0be87ef48ee4aff5478040a7d454ba7080dcca73ce8c10d91f WatchSource:0}: Error finding container c5a4cdfdf01f7b0be87ef48ee4aff5478040a7d454ba7080dcca73ce8c10d91f: Status 404 returned error can't find the container with id c5a4cdfdf01f7b0be87ef48ee4aff5478040a7d454ba7080dcca73ce8c10d91f Jul 18 01:56:54.800276 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:54.800252 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb31749ad92473fcf6d05f4eabaac6ec.slice/crio-28e0b5f7d68a63ea2ca560bc3168e57661d94ff2979e9895fe76e4c8316eee77 WatchSource:0}: Error finding container 28e0b5f7d68a63ea2ca560bc3168e57661d94ff2979e9895fe76e4c8316eee77: Status 404 returned error can't find the container with id 28e0b5f7d68a63ea2ca560bc3168e57661d94ff2979e9895fe76e4c8316eee77 Jul 18 01:56:54.801046 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:54.800944 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 01:56:54.804248 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:54.804224 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c6efe1e_41a0_429b_a45e_678499ab8f80.slice/crio-7a427e5dfa3b1b4fd8cde947af75778c4b34468d78ee39f87e4ea15b24a42857 WatchSource:0}: Error finding container 7a427e5dfa3b1b4fd8cde947af75778c4b34468d78ee39f87e4ea15b24a42857: Status 404 returned error can't find the container with id 7a427e5dfa3b1b4fd8cde947af75778c4b34468d78ee39f87e4ea15b24a42857 Jul 18 01:56:54.820622 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:54.820599 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce91eb83_04ec_4406_b92d_42beec0e867e.slice/crio-b613d5316147918254b5030b8972619407e99e051d6dfc50a4c6a358ad21f045 WatchSource:0}: Error finding container b613d5316147918254b5030b8972619407e99e051d6dfc50a4c6a358ad21f045: Status 404 returned error can't find the container with id b613d5316147918254b5030b8972619407e99e051d6dfc50a4c6a358ad21f045 Jul 18 01:56:55.032227 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:55.032079 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ff6170e_2dcc_4e43_b1d2_cac6fccc2339.slice/crio-e99bc3d18320ab704eec7bb662ba86f8714ae96a26b2b87be727f933a45374f8 WatchSource:0}: Error finding container e99bc3d18320ab704eec7bb662ba86f8714ae96a26b2b87be727f933a45374f8: Status 404 returned error can't find the container with id e99bc3d18320ab704eec7bb662ba86f8714ae96a26b2b87be727f933a45374f8 Jul 18 01:56:55.056600 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.056564 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:56:55.070888 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:55.070847 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83f2049b_7030_439f_843a_bda5238b11dc.slice/crio-b2649335695d7ee6811ee114eee02ed3cd2e5c1514e31f98e302a3ef0b23cdb9 WatchSource:0}: Error finding container b2649335695d7ee6811ee114eee02ed3cd2e5c1514e31f98e302a3ef0b23cdb9: Status 404 returned error can't find the container with id b2649335695d7ee6811ee114eee02ed3cd2e5c1514e31f98e302a3ef0b23cdb9 Jul 18 01:56:55.071545 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:55.071439 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6ca5e1_acf6_4e50_a04a_a0d039d44da9.slice/crio-6d5e32edd05c79e46274f89e8eece3cf6adbbc9b2e561174be0f5a343635c1e4 WatchSource:0}: Error finding container 6d5e32edd05c79e46274f89e8eece3cf6adbbc9b2e561174be0f5a343635c1e4: Status 404 returned error can't find the container with id 6d5e32edd05c79e46274f89e8eece3cf6adbbc9b2e561174be0f5a343635c1e4 Jul 18 01:56:55.082777 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.082744 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:55.082777 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.082777 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:55.082930 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.082876 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:56:55.082930 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.082878 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:56:55.082930 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.082924 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs podName:2992267b-15c9-4fa5-be5d-d783f560f9a6 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:56.082910906 +0000 UTC m=+3.871247467 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs") pod "network-metrics-daemon-zjvkm" (UID: "2992267b-15c9-4fa5-be5d-d783f560f9a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:56:55.083095 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.082936 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret podName:5c41bb40-6562-4a53-8641-8cb1b0a2c4f3 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:56.08293052 +0000 UTC m=+3.871267081 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret") pod "global-pull-secret-syncer-ljk5t" (UID: "5c41bb40-6562-4a53-8641-8cb1b0a2c4f3") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:56:55.096795 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:55.096759 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod288254b4_f459_4f1f_bb45_ae9180fbefa8.slice/crio-bf035a70e802b227269324fa9fd89f05e4f22f736a03db635b631713d7299676 WatchSource:0}: Error finding container bf035a70e802b227269324fa9fd89f05e4f22f736a03db635b631713d7299676: Status 404 returned error can't find the container with id bf035a70e802b227269324fa9fd89f05e4f22f736a03db635b631713d7299676 Jul 18 01:56:55.132218 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:56:55.132179 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1827c91_f990_469b_b0df_d612d539262c.slice/crio-78b9359267aaf8ebfef849f341fac14609916edcb62e0ba05487d9d0215ec747 WatchSource:0}: Error finding container 78b9359267aaf8ebfef849f341fac14609916edcb62e0ba05487d9d0215ec747: Status 404 returned error can't find the container with id 78b9359267aaf8ebfef849f341fac14609916edcb62e0ba05487d9d0215ec747 Jul 18 01:56:55.587019 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.586940 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkt4b\" (UniqueName: \"kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b\") pod \"network-check-target-m5dx8\" (UID: \"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5\") " pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:56:55.587210 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.587178 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:56:55.587210 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.587203 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:56:55.587327 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.587216 2573 projected.go:196] Error preparing data for projected volume kube-api-access-xkt4b for pod openshift-network-diagnostics/network-check-target-m5dx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:56:55.587327 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.587281 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b podName:009eb0e0-baa3-4a3a-94e3-2ce563f67bd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:57.587262243 +0000 UTC m=+5.375598824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xkt4b" (UniqueName: "kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b") pod "network-check-target-m5dx8" (UID: "009eb0e0-baa3-4a3a-94e3-2ce563f67bd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:56:55.607381 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.607340 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:51:53 +0000 UTC" deadline="2028-02-08 04:37:33.882716685 +0000 UTC" Jul 18 01:56:55.607381 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.607377 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13682h40m38.275345068s" Jul 18 01:56:55.623738 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.623678 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m2vjh" event={"ID":"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9","Type":"ContainerStarted","Data":"6d5e32edd05c79e46274f89e8eece3cf6adbbc9b2e561174be0f5a343635c1e4"} Jul 18 01:56:55.636447 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.636406 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" event={"ID":"83f2049b-7030-439f-843a-bda5238b11dc","Type":"ContainerStarted","Data":"b2649335695d7ee6811ee114eee02ed3cd2e5c1514e31f98e302a3ef0b23cdb9"} Jul 18 01:56:55.650926 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.650879 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" event={"ID":"ce91eb83-04ec-4406-b92d-42beec0e867e","Type":"ContainerStarted","Data":"b613d5316147918254b5030b8972619407e99e051d6dfc50a4c6a358ad21f045"} Jul 18 01:56:55.667589 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.667534 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fpf6q" event={"ID":"4c6efe1e-41a0-429b-a45e-678499ab8f80","Type":"ContainerStarted","Data":"7a427e5dfa3b1b4fd8cde947af75778c4b34468d78ee39f87e4ea15b24a42857"} Jul 18 01:56:55.671090 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.671057 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" event={"ID":"db31749ad92473fcf6d05f4eabaac6ec","Type":"ContainerStarted","Data":"28e0b5f7d68a63ea2ca560bc3168e57661d94ff2979e9895fe76e4c8316eee77"} Jul 18 01:56:55.680470 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.680403 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" event={"ID":"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8","Type":"ContainerStarted","Data":"c5a4cdfdf01f7b0be87ef48ee4aff5478040a7d454ba7080dcca73ce8c10d91f"} Jul 18 01:56:55.709422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.709381 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4h6fl" event={"ID":"087b9933-3cbe-42c2-bbc1-78326b899fe2","Type":"ContainerStarted","Data":"a7fa0ba492b55b2faf5dfcbb283b3cca9861e70d61a28b279abcec1b485f2f39"} Jul 18 01:56:55.716712 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.716683 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:55.716876 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.716819 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:56:55.717267 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.717247 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:56:55.717368 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.717347 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:56:55.717445 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.717419 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:55.717503 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:55.717492 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:56:55.735542 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.735282 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal" event={"ID":"e85115e7389ad24d665892c86f4108f1","Type":"ContainerStarted","Data":"0df06cebe796f2f928edb94f69c218b101b7fb578e6eb92468ef22b7db4cb29c"} Jul 18 01:56:55.767230 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.767145 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9fbkh" event={"ID":"f1827c91-f990-469b-b0df-d612d539262c","Type":"ContainerStarted","Data":"78b9359267aaf8ebfef849f341fac14609916edcb62e0ba05487d9d0215ec747"} Jul 18 01:56:55.807455 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.807339 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" event={"ID":"288254b4-f459-4f1f-bb45-ae9180fbefa8","Type":"ContainerStarted","Data":"bf035a70e802b227269324fa9fd89f05e4f22f736a03db635b631713d7299676"} Jul 18 01:56:55.835579 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:55.835296 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c6q2t" event={"ID":"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339","Type":"ContainerStarted","Data":"e99bc3d18320ab704eec7bb662ba86f8714ae96a26b2b87be727f933a45374f8"} Jul 18 01:56:56.018333 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:56.018066 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:56:56.091400 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:56.091370 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:56.091400 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:56.091405 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:56.091639 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:56.091504 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:56:56.091639 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:56.091554 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs podName:2992267b-15c9-4fa5-be5d-d783f560f9a6 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:58.091540913 +0000 UTC m=+5.879877474 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs") pod "network-metrics-daemon-zjvkm" (UID: "2992267b-15c9-4fa5-be5d-d783f560f9a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:56:56.091846 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:56.091830 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:56:56.091901 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:56.091873 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret podName:5c41bb40-6562-4a53-8641-8cb1b0a2c4f3 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:58.091863253 +0000 UTC m=+5.880199814 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret") pod "global-pull-secret-syncer-ljk5t" (UID: "5c41bb40-6562-4a53-8641-8cb1b0a2c4f3") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:56:57.604273 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:57.604229 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkt4b\" (UniqueName: \"kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b\") pod \"network-check-target-m5dx8\" (UID: \"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5\") " pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:56:57.604801 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:57.604469 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:56:57.604801 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:57.604491 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:56:57.604801 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:57.604504 2573 projected.go:196] Error preparing data for projected volume kube-api-access-xkt4b for pod openshift-network-diagnostics/network-check-target-m5dx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:56:57.604801 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:57.604565 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b podName:009eb0e0-baa3-4a3a-94e3-2ce563f67bd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:01.604547252 +0000 UTC m=+9.392883820 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xkt4b" (UniqueName: "kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b") pod "network-check-target-m5dx8" (UID: "009eb0e0-baa3-4a3a-94e3-2ce563f67bd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:56:57.717234 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:57.717199 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:57.717436 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:57.717367 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:56:57.717786 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:57.717765 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:56:57.717949 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:57.717927 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:56:57.718037 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:57.718015 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:57.718111 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:57.718093 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:56:58.108371 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:58.108336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:58.108551 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:58.108402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:58.108608 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:58.108573 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:56:58.108653 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:58.108638 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs podName:2992267b-15c9-4fa5-be5d-d783f560f9a6 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:02.108617904 +0000 UTC m=+9.896954484 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs") pod "network-metrics-daemon-zjvkm" (UID: "2992267b-15c9-4fa5-be5d-d783f560f9a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:56:58.109093 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:58.109054 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:56:58.109223 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:58.109120 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret podName:5c41bb40-6562-4a53-8641-8cb1b0a2c4f3 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:02.109105506 +0000 UTC m=+9.897442089 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret") pod "global-pull-secret-syncer-ljk5t" (UID: "5c41bb40-6562-4a53-8641-8cb1b0a2c4f3") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:56:59.716788 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:59.716689 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:56:59.716788 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:59.716720 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:56:59.717324 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:59.716843 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:56:59.717324 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:59.717250 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:56:59.717324 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:56:59.717289 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:56:59.717520 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:56:59.717358 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:01.641185 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:01.641094 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkt4b\" (UniqueName: \"kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b\") pod \"network-check-target-m5dx8\" (UID: \"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5\") " pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:01.641602 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:01.641297 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:57:01.641602 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:01.641324 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:57:01.641602 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:01.641338 2573 projected.go:196] Error preparing data for projected volume kube-api-access-xkt4b for pod openshift-network-diagnostics/network-check-target-m5dx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:57:01.641602 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:01.641407 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b podName:009eb0e0-baa3-4a3a-94e3-2ce563f67bd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:09.641387028 +0000 UTC m=+17.429723812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xkt4b" (UniqueName: "kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b") pod "network-check-target-m5dx8" (UID: "009eb0e0-baa3-4a3a-94e3-2ce563f67bd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:57:01.717264 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:01.716731 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:01.717264 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:01.716738 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:01.717264 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:01.716876 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:01.717264 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:01.716941 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:01.717264 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:01.716738 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:01.717264 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:01.717053 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:02.145866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:02.145822 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:02.145866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:02.145873 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:02.146148 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:02.146011 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:57:02.146148 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:02.146064 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:57:02.146148 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:02.146082 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret podName:5c41bb40-6562-4a53-8641-8cb1b0a2c4f3 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:10.146064891 +0000 UTC m=+17.934401453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret") pod "global-pull-secret-syncer-ljk5t" (UID: "5c41bb40-6562-4a53-8641-8cb1b0a2c4f3") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:57:02.146148 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:02.146139 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs podName:2992267b-15c9-4fa5-be5d-d783f560f9a6 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:10.146122161 +0000 UTC m=+17.934458729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs") pod "network-metrics-daemon-zjvkm" (UID: "2992267b-15c9-4fa5-be5d-d783f560f9a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:57:03.716822 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:03.716781 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:03.717296 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:03.716828 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:03.717296 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:03.716916 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:03.717296 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:03.716940 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:03.717296 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:03.717033 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:03.717296 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:03.717123 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:05.716848 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:05.716806 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:05.717327 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:05.716860 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:05.717327 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:05.716861 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:05.717327 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:05.716943 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:05.717327 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:05.717050 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:05.717327 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:05.717183 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:07.717412 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:07.717373 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:07.717412 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:07.717399 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:07.717865 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:07.717379 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:07.717865 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:07.717470 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:07.717865 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:07.717565 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:07.717865 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:07.717672 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:09.708906 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:09.708849 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkt4b\" (UniqueName: \"kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b\") pod \"network-check-target-m5dx8\" (UID: \"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5\") " pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:09.709411 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:09.709075 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:57:09.709411 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:09.709105 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:57:09.709411 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:09.709119 2573 projected.go:196] Error preparing data for projected volume kube-api-access-xkt4b for pod openshift-network-diagnostics/network-check-target-m5dx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:57:09.709411 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:09.709187 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b podName:009eb0e0-baa3-4a3a-94e3-2ce563f67bd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:25.709166288 +0000 UTC m=+33.497502861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xkt4b" (UniqueName: "kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b") pod "network-check-target-m5dx8" (UID: "009eb0e0-baa3-4a3a-94e3-2ce563f67bd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:57:09.716572 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:09.716539 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:09.716719 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:09.716547 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:09.716719 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:09.716654 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:09.716719 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:09.716555 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:09.716894 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:09.716711 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:09.716894 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:09.716821 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:10.213029 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:10.212984 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:10.213234 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:10.213039 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:10.213234 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:10.213214 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:57:10.213338 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:10.213243 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:57:10.213338 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:10.213300 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret podName:5c41bb40-6562-4a53-8641-8cb1b0a2c4f3 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:26.213280507 +0000 UTC m=+34.001617072 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret") pod "global-pull-secret-syncer-ljk5t" (UID: "5c41bb40-6562-4a53-8641-8cb1b0a2c4f3") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:57:10.213338 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:10.213324 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs podName:2992267b-15c9-4fa5-be5d-d783f560f9a6 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:26.213313124 +0000 UTC m=+34.001649692 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs") pod "network-metrics-daemon-zjvkm" (UID: "2992267b-15c9-4fa5-be5d-d783f560f9a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:57:11.717489 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:11.717399 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:11.717489 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:11.717445 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:11.717489 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:11.717445 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:11.718066 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:11.717536 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:11.718066 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:11.717642 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:11.718066 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:11.717736 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:13.717473 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:13.717433 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:13.717892 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:13.717433 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:13.717892 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:13.717554 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:13.717892 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:13.717433 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:13.717892 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:13.717658 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:13.717892 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:13.717771 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:14.891419 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:14.891380 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c6q2t" event={"ID":"1ff6170e-2dcc-4e43-b1d2-cac6fccc2339","Type":"ContainerStarted","Data":"75c162f5f8131cb56e12ae635e968a132da7c20ef2cbcd96165e7fa11523e24b"} Jul 18 01:57:14.892829 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:14.892801 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" event={"ID":"83f2049b-7030-439f-843a-bda5238b11dc","Type":"ContainerStarted","Data":"7f1f49af110cab79a8de58da6c120ab2a74b70891b6905f1ba7c34efe149278a"} Jul 18 01:57:14.894036 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:14.894014 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal" event={"ID":"e85115e7389ad24d665892c86f4108f1","Type":"ContainerStarted","Data":"08131238aa192f999d7d821de683ae584d5c835dffebf266a202548ff2627c21"} Jul 18 01:57:14.908602 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:14.908553 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-c6q2t" podStartSLOduration=3.192549721 podStartE2EDuration="22.908540156s" totalImagesPullingTime="19.715990435s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:14.908002547 +0000 UTC m=+22.696339208" watchObservedRunningTime="2026-07-18 01:57:14.908540156 +0000 UTC m=+22.696876738" Jul 18 01:57:14.922263 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:14.922211 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-187.ec2.internal" podStartSLOduration=21.922193706 podStartE2EDuration="21.922193706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:14.921914181 +0000 UTC m=+22.710250763" watchObservedRunningTime="2026-07-18 01:57:14.922193706 +0000 UTC m=+22.710530313" Jul 18 01:57:14.937512 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:14.937466 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jxvhp" podStartSLOduration=3.457506015 podStartE2EDuration="22.937450249s" totalImagesPullingTime="19.479944234s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:14.937449177 +0000 UTC m=+22.725785760" watchObservedRunningTime="2026-07-18 01:57:14.937450249 +0000 UTC m=+22.725786822" Jul 18 01:57:15.717704 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.717482 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:15.717902 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.717476 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:15.717902 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.717482 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:15.717902 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:15.717803 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:15.717902 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:15.717878 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:15.718136 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:15.717950 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:15.897854 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.897821 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" event={"ID":"ce91eb83-04ec-4406-b92d-42beec0e867e","Type":"ContainerStarted","Data":"e38bb381c35acc785317b9a45328faaf24f289f1c771192c5fb5769bdb036f16"} Jul 18 01:57:15.899119 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.899091 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fpf6q" event={"ID":"4c6efe1e-41a0-429b-a45e-678499ab8f80","Type":"ContainerStarted","Data":"e3b7d4d134c35c27063bb707affa96a98f1958510be2586e85e639105c750c35"} Jul 18 01:57:15.900426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.900406 2573 generic.go:356] "Generic (PLEG): container finished" podID="db31749ad92473fcf6d05f4eabaac6ec" containerID="9b412b485b1c64845c8dcf727a2e5a155be2cd4717478e6935b2030f34db4972" exitCode=0 Jul 18 01:57:15.900532 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.900476 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" event={"ID":"db31749ad92473fcf6d05f4eabaac6ec","Type":"ContainerDied","Data":"9b412b485b1c64845c8dcf727a2e5a155be2cd4717478e6935b2030f34db4972"} Jul 18 01:57:15.902937 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.902922 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 01:57:15.903241 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.903221 2573 generic.go:356] "Generic (PLEG): container finished" podID="2e4eb8b4-5a82-4673-a734-1a2d3842c1a8" containerID="74bf5477cf3a23aab01e4946df55566e3ee7f51525e2d36f6daffae42635a914" exitCode=1 Jul 18 01:57:15.903294 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.903282 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" event={"ID":"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8","Type":"ContainerStarted","Data":"746e9a6dfce1ddd314a18fc781c59cfbcf49f5674c30f866bcbf65188d4e356c"} Jul 18 01:57:15.903346 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.903337 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" event={"ID":"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8","Type":"ContainerStarted","Data":"df61738e54c113fecc06302e63bb6efee60be173c32277b85bfd1896e1e0db42"} Jul 18 01:57:15.903390 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.903378 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" event={"ID":"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8","Type":"ContainerStarted","Data":"0b38f59e032f95432bc7e24344ee1acadf2f318e29c2cfc5d740fa4d6ed348f4"} Jul 18 01:57:15.903426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.903418 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" event={"ID":"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8","Type":"ContainerStarted","Data":"3cb00e4b53a3d8bd8c18019de70b27041592049ff7b81524373bf1ac463a6046"} Jul 18 01:57:15.903463 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.903445 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" event={"ID":"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8","Type":"ContainerDied","Data":"74bf5477cf3a23aab01e4946df55566e3ee7f51525e2d36f6daffae42635a914"} Jul 18 01:57:15.903503 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.903484 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" event={"ID":"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8","Type":"ContainerStarted","Data":"5135e687109c84677f33b2a7c8b1dac11858773f4362f1868a4197cc0be2dcb5"} Jul 18 01:57:15.904544 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.904527 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9fbkh" event={"ID":"f1827c91-f990-469b-b0df-d612d539262c","Type":"ContainerStarted","Data":"685eb35605178ce715c05cfb786b7912dea78d73e5eadaeb2d8a7bcce0eaf007"} Jul 18 01:57:15.905998 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.905959 2573 generic.go:356] "Generic (PLEG): container finished" podID="288254b4-f459-4f1f-bb45-ae9180fbefa8" containerID="7ae47a548f0db19a1b74a43e2ea1744105749c237911b2be011593b5bb457f98" exitCode=0 Jul 18 01:57:15.906101 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.905999 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" event={"ID":"288254b4-f459-4f1f-bb45-ae9180fbefa8","Type":"ContainerDied","Data":"7ae47a548f0db19a1b74a43e2ea1744105749c237911b2be011593b5bb457f98"} Jul 18 01:57:15.907534 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.907513 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m2vjh" event={"ID":"3f6ca5e1-acf6-4e50-a04a-a0d039d44da9","Type":"ContainerStarted","Data":"bb268c7310b8a4456e07c57ba567402cf72205d4c5cc6de2c4880d787523e744"} Jul 18 01:57:15.936940 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.936884 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fpf6q" podStartSLOduration=4.215980384 podStartE2EDuration="23.93687205s" totalImagesPullingTime="19.720891666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:15.936708891 +0000 UTC m=+23.725045473" watchObservedRunningTime="2026-07-18 01:57:15.93687205 +0000 UTC m=+23.725208632" Jul 18 01:57:15.971877 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:15.971824 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m2vjh" podStartSLOduration=4.512794096 podStartE2EDuration="23.97180715s" totalImagesPullingTime="19.459013054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:15.971217825 +0000 UTC m=+23.759554398" watchObservedRunningTime="2026-07-18 01:57:15.97180715 +0000 UTC m=+23.760143731" Jul 18 01:57:16.625455 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:16.625427 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 18 01:57:16.675401 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:16.675305 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T01:57:16.625452242Z","UUID":"bafc9a38-9a67-4b35-b595-88264e1f1346","Handler":null,"Name":"","Endpoint":""} Jul 18 01:57:16.678240 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:16.678217 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 18 01:57:16.678418 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:16.678259 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 18 01:57:16.911261 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:16.911224 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" event={"ID":"ce91eb83-04ec-4406-b92d-42beec0e867e","Type":"ContainerStarted","Data":"8982caceaa98b8623d5e22b297662e3979c16ed7e2745a1d227be4398613fe8e"} Jul 18 01:57:16.913826 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:16.913790 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4h6fl" event={"ID":"087b9933-3cbe-42c2-bbc1-78326b899fe2","Type":"ContainerStarted","Data":"e2b0ebfddd0ecc98515b3880831c1c867090147a6149dd7126f72d03eb6dd3fa"} Jul 18 01:57:16.927390 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:16.927335 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9fbkh" podStartSLOduration=5.529029146 podStartE2EDuration="24.927315408s" totalImagesPullingTime="19.398286262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:16.077588831 +0000 UTC m=+23.865925413" watchObservedRunningTime="2026-07-18 01:57:16.927315408 +0000 UTC m=+24.715651990" Jul 18 01:57:16.927591 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:16.927555 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4h6fl" podStartSLOduration=5.170888211 podStartE2EDuration="24.927547716s" totalImagesPullingTime="19.756659505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:16.927266192 +0000 UTC m=+24.715602774" watchObservedRunningTime="2026-07-18 01:57:16.927547716 +0000 UTC m=+24.715884296" Jul 18 01:57:17.717229 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:17.717190 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:17.717467 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:17.717234 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:17.717467 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:17.717190 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:17.717467 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:17.717333 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:17.717467 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:17.717434 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:17.717672 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:17.717530 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:17.918140 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:17.918102 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" event={"ID":"db31749ad92473fcf6d05f4eabaac6ec","Type":"ContainerStarted","Data":"8f405b87078ede65c801445eccd612ce98b5d52b5212ef069bdf204f6aa78cc3"} Jul 18 01:57:17.921639 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:17.921620 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 01:57:17.922197 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:17.922173 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" event={"ID":"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8","Type":"ContainerStarted","Data":"c5e7bcbd5fa3cbffbc2fafacf5dba18d1a43b49ce7ec4f41a87ed66f08388139"} Jul 18 01:57:17.932568 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:17.932399 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-187.ec2.internal" podStartSLOduration=24.932375993 podStartE2EDuration="24.932375993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:17.931951345 +0000 UTC m=+25.720287927" watchObservedRunningTime="2026-07-18 01:57:17.932375993 +0000 UTC m=+25.720712576" Jul 18 01:57:18.927351 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:18.927307 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" event={"ID":"ce91eb83-04ec-4406-b92d-42beec0e867e","Type":"ContainerStarted","Data":"adf645358c4615a00106cbde747615d27a03c520f9943432f38f0bea524687f6"} Jul 18 01:57:18.946516 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:18.946453 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glx6v" podStartSLOduration=8.709060816000001 podStartE2EDuration="26.946437318s" totalImagesPullingTime="18.237376502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:18.94508447 +0000 UTC m=+26.733421054" watchObservedRunningTime="2026-07-18 01:57:18.946437318 +0000 UTC m=+26.734773919" Jul 18 01:57:19.717292 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:19.717256 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:19.717480 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:19.717256 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:19.717480 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:19.717387 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:19.717480 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:19.717267 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:19.717614 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:19.717432 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:19.717614 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:19.717549 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:20.467714 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.467480 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:57:20.468304 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.468164 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:57:20.935090 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.935064 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 01:57:20.935443 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.935418 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" event={"ID":"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8","Type":"ContainerStarted","Data":"ca2d8a76e4e77de1dde3e735b42198aba7325241fd32edf5d098777308a35f1d"} Jul 18 01:57:20.935741 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.935718 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:57:20.935881 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.935866 2573 scope.go:117] "RemoveContainer" containerID="74bf5477cf3a23aab01e4946df55566e3ee7f51525e2d36f6daffae42635a914" Jul 18 01:57:20.937413 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.937388 2573 generic.go:356] "Generic (PLEG): container finished" podID="288254b4-f459-4f1f-bb45-ae9180fbefa8" containerID="479336fe6e1973d23a10a59d28ad0fe60d0bcde7fb5566a6962929af34451256" exitCode=0 Jul 18 01:57:20.937524 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.937490 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" event={"ID":"288254b4-f459-4f1f-bb45-ae9180fbefa8","Type":"ContainerDied","Data":"479336fe6e1973d23a10a59d28ad0fe60d0bcde7fb5566a6962929af34451256"} Jul 18 01:57:20.937695 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.937674 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:57:20.938154 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.938136 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9fbkh" Jul 18 01:57:20.953418 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:20.953390 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:57:21.716866 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.716679 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:21.717471 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.716681 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:21.717471 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:21.716963 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:21.717471 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.716705 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:21.717471 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:21.717119 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:21.717471 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:21.717236 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:21.893401 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.893363 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:21.943500 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.943480 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 01:57:21.943882 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.943858 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" event={"ID":"2e4eb8b4-5a82-4673-a734-1a2d3842c1a8","Type":"ContainerStarted","Data":"33d0875147bea3ddd13cf2b08f5fd014a4e16e0268554edfc278c062cad9de91"} Jul 18 01:57:21.944164 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.944141 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:57:21.944288 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.944209 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:57:21.960714 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.960676 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:57:21.972099 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.971960 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" podStartSLOduration=9.791912877 podStartE2EDuration="29.971942952s" totalImagesPullingTime="20.180030075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:21.971671007 +0000 UTC m=+29.760007590" watchObservedRunningTime="2026-07-18 01:57:21.971942952 +0000 UTC m=+29.760279533" Jul 18 01:57:21.974499 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:21.974472 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:22.167368 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:22.167332 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ljk5t"] Jul 18 01:57:22.167632 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:22.167610 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:22.167756 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:22.167731 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:22.168019 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:22.167960 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zjvkm"] Jul 18 01:57:22.168100 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:22.168073 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:22.168168 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:22.168150 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:22.178416 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:22.178391 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m5dx8"] Jul 18 01:57:22.178533 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:22.178477 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:22.178573 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:22.178547 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:22.947602 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:22.947566 2573 generic.go:356] "Generic (PLEG): container finished" podID="288254b4-f459-4f1f-bb45-ae9180fbefa8" containerID="38da3b38bb21786bf8113ee3d85ac630eae6cc63ea828a48dd85212a495f7dd3" exitCode=0 Jul 18 01:57:22.948192 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:22.947644 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" event={"ID":"288254b4-f459-4f1f-bb45-ae9180fbefa8","Type":"ContainerDied","Data":"38da3b38bb21786bf8113ee3d85ac630eae6cc63ea828a48dd85212a495f7dd3"} Jul 18 01:57:23.717343 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:23.717303 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:23.717566 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:23.717348 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:23.717566 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:23.717303 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:23.717566 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:23.717416 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:23.717566 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:23.717488 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:23.717698 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:23.717564 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:24.954802 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:24.954601 2573 generic.go:356] "Generic (PLEG): container finished" podID="288254b4-f459-4f1f-bb45-ae9180fbefa8" containerID="40a872ea1930256ddc5ae105bfa1dd88d390d84b0df2f6e9eca21b6ca03eb848" exitCode=0 Jul 18 01:57:24.954802 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:24.954676 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" event={"ID":"288254b4-f459-4f1f-bb45-ae9180fbefa8","Type":"ContainerDied","Data":"40a872ea1930256ddc5ae105bfa1dd88d390d84b0df2f6e9eca21b6ca03eb848"} Jul 18 01:57:24.963353 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:24.963327 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:25.037241 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:25.037205 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:25.137562 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:25.137516 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:25.237799 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:25.237714 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:25.295696 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:25.295659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:25.446788 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:25.446753 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:25.717360 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:25.717329 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:25.717360 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:25.717364 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:25.717591 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:25.717330 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:25.717591 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:25.717460 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:25.717591 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:25.717564 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:25.717740 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:25.717658 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:25.733127 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:25.733096 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkt4b\" (UniqueName: \"kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b\") pod \"network-check-target-m5dx8\" (UID: \"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5\") " pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:25.733301 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:25.733236 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:57:25.733301 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:25.733256 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:57:25.733301 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:25.733267 2573 projected.go:196] Error preparing data for projected volume kube-api-access-xkt4b for pod openshift-network-diagnostics/network-check-target-m5dx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:57:25.733436 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:25.733324 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b podName:009eb0e0-baa3-4a3a-94e3-2ce563f67bd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:57.733306659 +0000 UTC m=+65.521643223 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xkt4b" (UniqueName: "kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b") pod "network-check-target-m5dx8" (UID: "009eb0e0-baa3-4a3a-94e3-2ce563f67bd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:57:26.236067 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:26.236030 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:26.236466 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:26.236080 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:26.236466 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:26.236146 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:57:26.236466 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:26.236166 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:57:26.236466 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:26.236213 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs podName:2992267b-15c9-4fa5-be5d-d783f560f9a6 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:58.236194832 +0000 UTC m=+66.024531409 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs") pod "network-metrics-daemon-zjvkm" (UID: "2992267b-15c9-4fa5-be5d-d783f560f9a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:57:26.236466 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:26.236226 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret podName:5c41bb40-6562-4a53-8641-8cb1b0a2c4f3 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:58.236220475 +0000 UTC m=+66.024557035 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret") pod "global-pull-secret-syncer-ljk5t" (UID: "5c41bb40-6562-4a53-8641-8cb1b0a2c4f3") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:57:26.389066 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:26.389025 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:26.489322 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:26.489283 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:27.717126 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:27.717072 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:27.717126 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:27.717102 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:27.717669 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:27.717083 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:27.717669 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:27.717207 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjvkm" podUID="2992267b-15c9-4fa5-be5d-d783f560f9a6" Jul 18 01:57:27.717669 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:27.717258 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5dx8" podUID="009eb0e0-baa3-4a3a-94e3-2ce563f67bd5" Jul 18 01:57:27.717669 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:27.717322 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljk5t" podUID="5c41bb40-6562-4a53-8641-8cb1b0a2c4f3" Jul 18 01:57:28.018796 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.018769 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-187.ec2.internal" event="NodeReady" Jul 18 01:57:28.018997 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.018912 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 18 01:57:28.054707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.054674 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67d8b55cdb-26qkq"] Jul 18 01:57:28.079837 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.079806 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q6gs9"] Jul 18 01:57:28.080018 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.079951 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.083479 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.083448 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 18 01:57:28.083722 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.083681 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 18 01:57:28.083722 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.083696 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-q7jhh\"" Jul 18 01:57:28.083934 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.083906 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 18 01:57:28.099744 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.099716 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j9bdq"] Jul 18 01:57:28.100465 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.100154 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.102309 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.102292 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 18 01:57:28.102788 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.102771 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 18 01:57:28.102878 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.102778 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-96mg4\"" Jul 18 01:57:28.104752 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.104735 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 18 01:57:28.113698 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.113675 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67d8b55cdb-26qkq"] Jul 18 01:57:28.113821 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.113709 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q6gs9"] Jul 18 01:57:28.113821 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.113742 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j9bdq"] Jul 18 01:57:28.113821 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.113821 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j9bdq" Jul 18 01:57:28.116586 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.116570 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 18 01:57:28.116699 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.116610 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 18 01:57:28.116850 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.116833 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kcv7c\"" Jul 18 01:57:28.117027 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.117009 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 18 01:57:28.149840 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.149801 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.150023 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.149849 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-bound-sa-token\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.150023 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.149880 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-installation-pull-secrets\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.150023 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.150011 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj8fd\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-kube-api-access-bj8fd\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.150201 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.150053 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-trusted-ca\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.150201 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.150100 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-image-registry-private-configuration\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.150201 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.150128 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-ca-trust-extracted\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.150201 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.150152 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-certificates\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.226430 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.226387 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:28.251298 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251257 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj8fd\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-kube-api-access-bj8fd\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.251298 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251302 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c0625e69-2b51-4675-9080-fc196d8d8070-config-volume\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.251551 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251327 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0625e69-2b51-4675-9080-fc196d8d8070-tmp-dir\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.251551 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l57sg\" (UniqueName: \"kubernetes.io/projected/c0625e69-2b51-4675-9080-fc196d8d8070-kube-api-access-l57sg\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.251551 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251500 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-trusted-ca\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.251551 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251541 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-image-registry-private-configuration\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.251739 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251572 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-ca-trust-extracted\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.251739 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251597 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-certificates\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.251739 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251654 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.251879 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:28.251745 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:57:28.251879 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:28.251760 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67d8b55cdb-26qkq: secret "image-registry-tls" not found Jul 18 01:57:28.251879 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251820 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.251879 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:28.251832 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls podName:08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:28.751810899 +0000 UTC m=+36.540147477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls") pod "image-registry-67d8b55cdb-26qkq" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1") : secret "image-registry-tls" not found Jul 18 01:57:28.251879 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251868 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtk5p\" (UniqueName: \"kubernetes.io/projected/25808b10-82d5-4db1-867a-7791b405f163-kube-api-access-qtk5p\") pod \"ingress-canary-j9bdq\" (UID: \"25808b10-82d5-4db1-867a-7791b405f163\") " pod="openshift-ingress-canary/ingress-canary-j9bdq" Jul 18 01:57:28.252151 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-bound-sa-token\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.252151 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251942 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-installation-pull-secrets\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.252151 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.251958 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25808b10-82d5-4db1-867a-7791b405f163-cert\") pod \"ingress-canary-j9bdq\" (UID: \"25808b10-82d5-4db1-867a-7791b405f163\") " pod="openshift-ingress-canary/ingress-canary-j9bdq" Jul 18 01:57:28.252305 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.252272 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-ca-trust-extracted\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.252305 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.252296 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-certificates\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.252604 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.252584 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-trusted-ca\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.256237 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.256219 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-image-registry-private-configuration\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.256340 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.256223 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-installation-pull-secrets\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.262065 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.262041 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj8fd\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-kube-api-access-bj8fd\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.262204 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.262047 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-bound-sa-token\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.352924 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.352824 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.352924 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:28.352857 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:57:28.352924 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.352906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtk5p\" (UniqueName: \"kubernetes.io/projected/25808b10-82d5-4db1-867a-7791b405f163-kube-api-access-qtk5p\") pod \"ingress-canary-j9bdq\" (UID: \"25808b10-82d5-4db1-867a-7791b405f163\") " pod="openshift-ingress-canary/ingress-canary-j9bdq" Jul 18 01:57:28.353215 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:28.352941 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls podName:c0625e69-2b51-4675-9080-fc196d8d8070 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:28.852920825 +0000 UTC m=+36.641257390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls") pod "dns-default-q6gs9" (UID: "c0625e69-2b51-4675-9080-fc196d8d8070") : secret "dns-default-metrics-tls" not found Jul 18 01:57:28.353215 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.352994 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25808b10-82d5-4db1-867a-7791b405f163-cert\") pod \"ingress-canary-j9bdq\" (UID: \"25808b10-82d5-4db1-867a-7791b405f163\") " pod="openshift-ingress-canary/ingress-canary-j9bdq" Jul 18 01:57:28.353215 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.353034 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c0625e69-2b51-4675-9080-fc196d8d8070-config-volume\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.353215 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.353063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0625e69-2b51-4675-9080-fc196d8d8070-tmp-dir\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.353215 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.353173 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l57sg\" (UniqueName: \"kubernetes.io/projected/c0625e69-2b51-4675-9080-fc196d8d8070-kube-api-access-l57sg\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.353478 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.353452 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0625e69-2b51-4675-9080-fc196d8d8070-tmp-dir\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.353793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.353771 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c0625e69-2b51-4675-9080-fc196d8d8070-config-volume\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.355668 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.355646 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25808b10-82d5-4db1-867a-7791b405f163-cert\") pod \"ingress-canary-j9bdq\" (UID: \"25808b10-82d5-4db1-867a-7791b405f163\") " pod="openshift-ingress-canary/ingress-canary-j9bdq" Jul 18 01:57:28.368741 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.368710 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l57sg\" (UniqueName: \"kubernetes.io/projected/c0625e69-2b51-4675-9080-fc196d8d8070-kube-api-access-l57sg\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.368741 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.368731 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtk5p\" (UniqueName: \"kubernetes.io/projected/25808b10-82d5-4db1-867a-7791b405f163-kube-api-access-qtk5p\") pod \"ingress-canary-j9bdq\" (UID: \"25808b10-82d5-4db1-867a-7791b405f163\") " pod="openshift-ingress-canary/ingress-canary-j9bdq" Jul 18 01:57:28.422991 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.422935 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j9bdq" Jul 18 01:57:28.440793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.440755 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:28.552792 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.552762 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:28.651603 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.651565 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:28.756357 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.756320 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:28.757017 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:28.756502 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:57:28.757017 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:28.756522 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67d8b55cdb-26qkq: secret "image-registry-tls" not found Jul 18 01:57:28.757017 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:28.756610 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls podName:08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:29.756580085 +0000 UTC m=+37.544916646 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls") pod "image-registry-67d8b55cdb-26qkq" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1") : secret "image-registry-tls" not found Jul 18 01:57:28.857266 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:28.857227 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:28.857448 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:28.857426 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:57:28.857528 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:28.857517 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls podName:c0625e69-2b51-4675-9080-fc196d8d8070 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:29.857495881 +0000 UTC m=+37.645832457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls") pod "dns-default-q6gs9" (UID: "c0625e69-2b51-4675-9080-fc196d8d8070") : secret "dns-default-metrics-tls" not found Jul 18 01:57:29.209815 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.209791 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j9bdq"] Jul 18 01:57:29.240982 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:57:29.240944 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25808b10_82d5_4db1_867a_7791b405f163.slice/crio-68f8eeb4ca4299c3876822347f6cb2c671b1ee4dcffb583c9a71d8f61e4f1fe2 WatchSource:0}: Error finding container 68f8eeb4ca4299c3876822347f6cb2c671b1ee4dcffb583c9a71d8f61e4f1fe2: Status 404 returned error can't find the container with id 68f8eeb4ca4299c3876822347f6cb2c671b1ee4dcffb583c9a71d8f61e4f1fe2 Jul 18 01:57:29.497006 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.496787 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:29.598368 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.598279 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:29.716815 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.716780 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:29.717019 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.716780 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:29.717346 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.717130 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:29.719652 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.719629 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 01:57:29.719652 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.719647 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 01:57:29.720998 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.720458 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 01:57:29.720998 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.720462 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 01:57:29.720998 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.720554 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zr2dl\"" Jul 18 01:57:29.720998 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.720806 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5xbv\"" Jul 18 01:57:29.764962 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.764930 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:29.765493 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:29.765115 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:57:29.765493 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:29.765138 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67d8b55cdb-26qkq: secret "image-registry-tls" not found Jul 18 01:57:29.765493 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:29.765196 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls podName:08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:31.76517846 +0000 UTC m=+39.553515020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls") pod "image-registry-67d8b55cdb-26qkq" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1") : secret "image-registry-tls" not found Jul 18 01:57:29.866366 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.866280 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:29.866534 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:29.866468 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:57:29.866593 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:29.866559 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls podName:c0625e69-2b51-4675-9080-fc196d8d8070 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:31.86653497 +0000 UTC m=+39.654871549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls") pod "dns-default-q6gs9" (UID: "c0625e69-2b51-4675-9080-fc196d8d8070") : secret "dns-default-metrics-tls" not found Jul 18 01:57:29.969446 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.969414 2573 generic.go:356] "Generic (PLEG): container finished" podID="288254b4-f459-4f1f-bb45-ae9180fbefa8" containerID="0763be6df444b3cd0547b126ab36e4f189db48a467941f468200c65156dd3609" exitCode=0 Jul 18 01:57:29.969632 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.969496 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" event={"ID":"288254b4-f459-4f1f-bb45-ae9180fbefa8","Type":"ContainerDied","Data":"0763be6df444b3cd0547b126ab36e4f189db48a467941f468200c65156dd3609"} Jul 18 01:57:29.970796 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:29.970772 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j9bdq" event={"ID":"25808b10-82d5-4db1-867a-7791b405f163","Type":"ContainerStarted","Data":"68f8eeb4ca4299c3876822347f6cb2c671b1ee4dcffb583c9a71d8f61e4f1fe2"} Jul 18 01:57:30.436305 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:30.436258 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:30.977879 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:30.977842 2573 generic.go:356] "Generic (PLEG): container finished" podID="288254b4-f459-4f1f-bb45-ae9180fbefa8" containerID="e5047379eeaba7cd301b4ae6d01490c3ef1eee39819ce47a23c8c5320034eb00" exitCode=0 Jul 18 01:57:30.978372 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:30.977905 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" event={"ID":"288254b4-f459-4f1f-bb45-ae9180fbefa8","Type":"ContainerDied","Data":"e5047379eeaba7cd301b4ae6d01490c3ef1eee39819ce47a23c8c5320034eb00"} Jul 18 01:57:31.783599 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:31.783426 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:31.783736 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:31.783629 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:57:31.783736 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:31.783654 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67d8b55cdb-26qkq: secret "image-registry-tls" not found Jul 18 01:57:31.783802 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:31.783735 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls podName:08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:35.783711639 +0000 UTC m=+43.572048222 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls") pod "image-registry-67d8b55cdb-26qkq" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1") : secret "image-registry-tls" not found Jul 18 01:57:31.884562 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:31.884523 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:31.884741 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:31.884637 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:57:31.884741 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:31.884702 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls podName:c0625e69-2b51-4675-9080-fc196d8d8070 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:35.884687709 +0000 UTC m=+43.673024269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls") pod "dns-default-q6gs9" (UID: "c0625e69-2b51-4675-9080-fc196d8d8070") : secret "dns-default-metrics-tls" not found Jul 18 01:57:31.885521 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:31.885493 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:31.983385 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:31.983348 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" event={"ID":"288254b4-f459-4f1f-bb45-ae9180fbefa8","Type":"ContainerStarted","Data":"0e0b9cbfd0c77ee87baf422defd0f80832b9cb332bd48eb5a144d6c6ae6a0d35"} Jul 18 01:57:31.984738 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:31.984717 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j9bdq" event={"ID":"25808b10-82d5-4db1-867a-7791b405f163","Type":"ContainerStarted","Data":"96e22cde9c92760bdbee7806f60fb795ecf2c7634eb8913b6d4d0ad154ce5533"} Jul 18 01:57:31.986948 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:31.986921 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:32.005100 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:32.004963 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rg7tx" podStartSLOduration=9.596644062 podStartE2EDuration="40.004945716s" totalImagesPullingTime="30.408301654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:32.004613134 +0000 UTC m=+39.792949715" watchObservedRunningTime="2026-07-18 01:57:32.004945716 +0000 UTC m=+39.793282299" Jul 18 01:57:32.018946 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:32.018901 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j9bdq" podStartSLOduration=1.9799572159999999 podStartE2EDuration="4.018887396s" totalImagesPullingTime="2.03893018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:32.018545117 +0000 UTC m=+39.806881703" watchObservedRunningTime="2026-07-18 01:57:32.018887396 +0000 UTC m=+39.807223992" Jul 18 01:57:32.187235 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:32.187193 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:32.240651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:32.240619 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:33.464583 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:33.464552 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:33.517070 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:33.517015 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:34.137519 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.137488 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zjfnx"] Jul 18 01:57:34.139756 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.139738 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.142109 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.142076 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 18 01:57:34.142220 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.142136 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 18 01:57:34.142271 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.142221 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 18 01:57:34.142324 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.142309 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 18 01:57:34.143334 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.143321 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rd42x\"" Jul 18 01:57:34.149447 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.149422 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zjfnx"] Jul 18 01:57:34.172790 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.172763 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:34.228871 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.228837 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt"] Jul 18 01:57:34.230724 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.230709 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt" Jul 18 01:57:34.232876 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.232853 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 18 01:57:34.232876 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.232868 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wrfts\"" Jul 18 01:57:34.233087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.232873 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 18 01:57:34.239454 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.239437 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt"] Jul 18 01:57:34.262532 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.262499 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:34.303506 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.303469 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1abbff1-f365-4771-a560-10fee79a4ea2-data-volume\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.303506 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.303507 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.303694 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.303538 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt5vf\" (UniqueName: \"kubernetes.io/projected/d797431d-6fc4-47b6-8eca-5eeb7d664185-kube-api-access-vt5vf\") pod \"migrator-7cc9c7ffd7-2xgqt\" (UID: \"d797431d-6fc4-47b6-8eca-5eeb7d664185\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt" Jul 18 01:57:34.303694 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.303651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1abbff1-f365-4771-a560-10fee79a4ea2-crio-socket\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.303694 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.303676 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.303780 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.303713 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hps8\" (UniqueName: \"kubernetes.io/projected/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-api-access-9hps8\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.317434 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.317399 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:34.404814 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.404732 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1abbff1-f365-4771-a560-10fee79a4ea2-data-volume\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.404814 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.404770 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.404814 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.404789 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt5vf\" (UniqueName: \"kubernetes.io/projected/d797431d-6fc4-47b6-8eca-5eeb7d664185-kube-api-access-vt5vf\") pod \"migrator-7cc9c7ffd7-2xgqt\" (UID: \"d797431d-6fc4-47b6-8eca-5eeb7d664185\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt" Jul 18 01:57:34.405081 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.404864 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1abbff1-f365-4771-a560-10fee79a4ea2-crio-socket\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.405081 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:34.404871 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:57:34.405081 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.404890 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.405081 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:34.404944 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm podName:d1abbff1-f365-4771-a560-10fee79a4ea2 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:34.904925885 +0000 UTC m=+42.693262468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zjfnx" (UID: "d1abbff1-f365-4771-a560-10fee79a4ea2") : configmap "kube-rbac-proxy" not found Jul 18 01:57:34.405081 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:34.405014 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:57:34.405081 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:34.405060 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls podName:d1abbff1-f365-4771-a560-10fee79a4ea2 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:34.905048384 +0000 UTC m=+42.693384946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zjfnx" (UID: "d1abbff1-f365-4771-a560-10fee79a4ea2") : secret "insights-runtime-extractor-tls" not found Jul 18 01:57:34.405350 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.404962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hps8\" (UniqueName: \"kubernetes.io/projected/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-api-access-9hps8\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.405350 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.405139 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1abbff1-f365-4771-a560-10fee79a4ea2-data-volume\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.405350 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.405148 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1abbff1-f365-4771-a560-10fee79a4ea2-crio-socket\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.423381 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.423349 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hps8\" (UniqueName: \"kubernetes.io/projected/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-api-access-9hps8\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.423502 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.423349 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt5vf\" (UniqueName: \"kubernetes.io/projected/d797431d-6fc4-47b6-8eca-5eeb7d664185-kube-api-access-vt5vf\") pod \"migrator-7cc9c7ffd7-2xgqt\" (UID: \"d797431d-6fc4-47b6-8eca-5eeb7d664185\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt" Jul 18 01:57:34.495100 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.495066 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:34.538548 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.538519 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt" Jul 18 01:57:34.546184 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.546156 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:34.662890 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.662819 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt"] Jul 18 01:57:34.666265 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:57:34.666233 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd797431d_6fc4_47b6_8eca_5eeb7d664185.slice/crio-e7032de4470bbef6cd02054388a70fdf159fb7707785e464cf9f8d40f28bccf0 WatchSource:0}: Error finding container e7032de4470bbef6cd02054388a70fdf159fb7707785e464cf9f8d40f28bccf0: Status 404 returned error can't find the container with id e7032de4470bbef6cd02054388a70fdf159fb7707785e464cf9f8d40f28bccf0 Jul 18 01:57:34.908664 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.908626 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.908833 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.908692 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:34.908833 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:34.908769 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:57:34.908833 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:34.908793 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:57:34.908833 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:34.908828 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls podName:d1abbff1-f365-4771-a560-10fee79a4ea2 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:35.908813958 +0000 UTC m=+43.697150518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zjfnx" (UID: "d1abbff1-f365-4771-a560-10fee79a4ea2") : secret "insights-runtime-extractor-tls" not found Jul 18 01:57:34.909021 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:34.908843 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm podName:d1abbff1-f365-4771-a560-10fee79a4ea2 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:35.908836744 +0000 UTC m=+43.697173304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zjfnx" (UID: "d1abbff1-f365-4771-a560-10fee79a4ea2") : configmap "kube-rbac-proxy" not found Jul 18 01:57:34.928171 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.928095 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:34.991864 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:34.991835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt" event={"ID":"d797431d-6fc4-47b6-8eca-5eeb7d664185","Type":"ContainerStarted","Data":"e7032de4470bbef6cd02054388a70fdf159fb7707785e464cf9f8d40f28bccf0"} Jul 18 01:57:35.028007 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:35.027952 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:35.204754 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:35.204670 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:35.304891 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:35.304804 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:35.723833 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:35.723707 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fpf6q_4c6efe1e-41a0-429b-a45e-678499ab8f80/dns-node-resolver/0.log" Jul 18 01:57:35.816304 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:35.816265 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:35.816484 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:35.816443 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:57:35.816527 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:35.816486 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67d8b55cdb-26qkq: secret "image-registry-tls" not found Jul 18 01:57:35.816556 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:35.816545 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls podName:08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:43.81652892 +0000 UTC m=+51.604865483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls") pod "image-registry-67d8b55cdb-26qkq" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1") : secret "image-registry-tls" not found Jul 18 01:57:35.917120 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:35.917086 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:35.917120 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:35.917122 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:35.917341 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:35.917162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:35.917341 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:35.917254 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:57:35.917341 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:35.917257 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:57:35.917341 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:35.917266 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:57:35.917341 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:35.917301 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm podName:d1abbff1-f365-4771-a560-10fee79a4ea2 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:37.917288759 +0000 UTC m=+45.705625320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zjfnx" (UID: "d1abbff1-f365-4771-a560-10fee79a4ea2") : configmap "kube-rbac-proxy" not found Jul 18 01:57:35.917341 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:35.917321 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls podName:c0625e69-2b51-4675-9080-fc196d8d8070 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:43.917308102 +0000 UTC m=+51.705644675 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls") pod "dns-default-q6gs9" (UID: "c0625e69-2b51-4675-9080-fc196d8d8070") : secret "dns-default-metrics-tls" not found Jul 18 01:57:35.917341 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:35.917335 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls podName:d1abbff1-f365-4771-a560-10fee79a4ea2 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:37.917329472 +0000 UTC m=+45.705666032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zjfnx" (UID: "d1abbff1-f365-4771-a560-10fee79a4ea2") : secret "insights-runtime-extractor-tls" not found Jul 18 01:57:36.056471 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.056392 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:36.112070 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.112036 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:36.797142 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.797106 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-xncg2"] Jul 18 01:57:36.822358 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.822327 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-xncg2"] Jul 18 01:57:36.822516 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.822422 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:36.824727 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.824701 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 18 01:57:36.824863 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.824819 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 18 01:57:36.824942 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.824928 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 18 01:57:36.825707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.825685 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 18 01:57:36.825819 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.825753 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zq7n5\"" Jul 18 01:57:36.833149 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.833110 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:36.888808 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.888770 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:36.922678 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.922651 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m2vjh_3f6ca5e1-acf6-4e50-a04a-a0d039d44da9/node-ca/0.log" Jul 18 01:57:36.927054 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.927031 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fa3d41ad-ea73-43d2-a841-4ea136020bff-signing-key\") pod \"service-ca-58598695f7-xncg2\" (UID: \"fa3d41ad-ea73-43d2-a841-4ea136020bff\") " pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:36.927157 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.927087 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fa3d41ad-ea73-43d2-a841-4ea136020bff-signing-cabundle\") pod \"service-ca-58598695f7-xncg2\" (UID: \"fa3d41ad-ea73-43d2-a841-4ea136020bff\") " pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:36.927157 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.927113 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pl76\" (UniqueName: \"kubernetes.io/projected/fa3d41ad-ea73-43d2-a841-4ea136020bff-kube-api-access-5pl76\") pod \"service-ca-58598695f7-xncg2\" (UID: \"fa3d41ad-ea73-43d2-a841-4ea136020bff\") " pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:36.998513 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.998479 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt" event={"ID":"d797431d-6fc4-47b6-8eca-5eeb7d664185","Type":"ContainerStarted","Data":"053cf91627a3189adaa16bdfe10c57517b34f76869825bc7a9f1811fa6914825"} Jul 18 01:57:36.998697 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:36.998529 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt" event={"ID":"d797431d-6fc4-47b6-8eca-5eeb7d664185","Type":"ContainerStarted","Data":"c6404f651b3e164589f1df78af71173e50d956f000022ae1fc9919902c4a5d82"} Jul 18 01:57:37.014187 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.014138 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2xgqt" podStartSLOduration=1.494378811 podStartE2EDuration="3.014123774s" totalImagesPullingTime="1.519744963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:37.012828958 +0000 UTC m=+44.801165550" watchObservedRunningTime="2026-07-18 01:57:37.014123774 +0000 UTC m=+44.802460355" Jul 18 01:57:37.027631 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.027601 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fa3d41ad-ea73-43d2-a841-4ea136020bff-signing-cabundle\") pod \"service-ca-58598695f7-xncg2\" (UID: \"fa3d41ad-ea73-43d2-a841-4ea136020bff\") " pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:37.027714 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.027636 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pl76\" (UniqueName: \"kubernetes.io/projected/fa3d41ad-ea73-43d2-a841-4ea136020bff-kube-api-access-5pl76\") pod \"service-ca-58598695f7-xncg2\" (UID: \"fa3d41ad-ea73-43d2-a841-4ea136020bff\") " pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:37.027714 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.027710 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fa3d41ad-ea73-43d2-a841-4ea136020bff-signing-key\") pod \"service-ca-58598695f7-xncg2\" (UID: \"fa3d41ad-ea73-43d2-a841-4ea136020bff\") " pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:37.028257 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.028237 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fa3d41ad-ea73-43d2-a841-4ea136020bff-signing-cabundle\") pod \"service-ca-58598695f7-xncg2\" (UID: \"fa3d41ad-ea73-43d2-a841-4ea136020bff\") " pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:37.029942 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.029926 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fa3d41ad-ea73-43d2-a841-4ea136020bff-signing-key\") pod \"service-ca-58598695f7-xncg2\" (UID: \"fa3d41ad-ea73-43d2-a841-4ea136020bff\") " pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:37.035711 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.035689 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pl76\" (UniqueName: \"kubernetes.io/projected/fa3d41ad-ea73-43d2-a841-4ea136020bff-kube-api-access-5pl76\") pod \"service-ca-58598695f7-xncg2\" (UID: \"fa3d41ad-ea73-43d2-a841-4ea136020bff\") " pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:37.102367 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.102262 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:37.131286 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.131248 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-xncg2" Jul 18 01:57:37.160885 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.160854 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:37.290904 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.290873 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-xncg2"] Jul 18 01:57:37.295861 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:57:37.295825 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3d41ad_ea73_43d2_a841_4ea136020bff.slice/crio-9b021b1d029e857de2db1a5c61f6adc9c06715b84573781c855f57a6c39c9136 WatchSource:0}: Error finding container 9b021b1d029e857de2db1a5c61f6adc9c06715b84573781c855f57a6c39c9136: Status 404 returned error can't find the container with id 9b021b1d029e857de2db1a5c61f6adc9c06715b84573781c855f57a6c39c9136 Jul 18 01:57:37.323603 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.323579 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j9bdq_25808b10-82d5-4db1-867a-7791b405f163/serve-healthcheck-canary/0.log" Jul 18 01:57:37.703563 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.703531 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:37.935138 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.935098 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:37.935564 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.935224 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:37.935564 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:37.935376 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:57:37.935564 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:37.935484 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls podName:d1abbff1-f365-4771-a560-10fee79a4ea2 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:41.935462407 +0000 UTC m=+49.723798981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zjfnx" (UID: "d1abbff1-f365-4771-a560-10fee79a4ea2") : secret "insights-runtime-extractor-tls" not found Jul 18 01:57:37.935731 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:37.935712 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1abbff1-f365-4771-a560-10fee79a4ea2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:38.003134 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:38.003048 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-xncg2" event={"ID":"fa3d41ad-ea73-43d2-a841-4ea136020bff","Type":"ContainerStarted","Data":"9b021b1d029e857de2db1a5c61f6adc9c06715b84573781c855f57a6c39c9136"} Jul 18 01:57:40.010013 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:40.009958 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-xncg2" event={"ID":"fa3d41ad-ea73-43d2-a841-4ea136020bff","Type":"ContainerStarted","Data":"72daceeaa0785fd9a5ab14ef533b44c60122a4cd9f5ebd07409b0c8b1d542e18"} Jul 18 01:57:40.027187 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:40.027136 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-xncg2" podStartSLOduration=1.961578905 podStartE2EDuration="4.027117887s" totalImagesPullingTime="2.065538982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:40.026429091 +0000 UTC m=+47.814765672" watchObservedRunningTime="2026-07-18 01:57:40.027117887 +0000 UTC m=+47.815454507" Jul 18 01:57:40.383171 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:40.383078 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:40.450067 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:40.450033 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:40.523874 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:40.523835 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:40.639999 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:40.639888 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:40.725373 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:40.725337 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:41.494745 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:41.494709 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:41.885247 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:41.885194 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:41.972400 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:41.972362 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:41.972608 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:41.972497 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:57:41.972608 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:41.972564 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls podName:d1abbff1-f365-4771-a560-10fee79a4ea2 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:49.972547163 +0000 UTC m=+57.760883741 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zjfnx" (UID: "d1abbff1-f365-4771-a560-10fee79a4ea2") : secret "insights-runtime-extractor-tls" not found Jul 18 01:57:42.288666 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:42.288558 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:42.689192 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:42.689157 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:42.888499 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:42.888466 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:43.291386 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:43.291356 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:43.684809 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:43.684774 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:43.787164 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:43.787104 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:43.889220 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:43.889180 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:43.889385 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:43.889324 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:57:43.889385 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:43.889343 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67d8b55cdb-26qkq: secret "image-registry-tls" not found Jul 18 01:57:43.889450 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:57:43.889397 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls podName:08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:59.889381155 +0000 UTC m=+67.677717717 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls") pod "image-registry-67d8b55cdb-26qkq" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1") : secret "image-registry-tls" not found Jul 18 01:57:43.914399 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:43.914365 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:43.989588 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:43.989506 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:43.991727 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:43.991700 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0625e69-2b51-4675-9080-fc196d8d8070-metrics-tls\") pod \"dns-default-q6gs9\" (UID: \"c0625e69-2b51-4675-9080-fc196d8d8070\") " pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:44.010915 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:44.010882 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:44.140801 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:44.140770 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q6gs9"] Jul 18 01:57:44.144227 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:57:44.144195 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0625e69_2b51_4675_9080_fc196d8d8070.slice/crio-69c0fb80e35bb9d365f30aaa259973ddf88420fbec5125a59feb337e1fa4d247 WatchSource:0}: Error finding container 69c0fb80e35bb9d365f30aaa259973ddf88420fbec5125a59feb337e1fa4d247: Status 404 returned error can't find the container with id 69c0fb80e35bb9d365f30aaa259973ddf88420fbec5125a59feb337e1fa4d247 Jul 18 01:57:44.231217 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:44.231179 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:44.491500 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:44.491465 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:44.689753 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:44.689269 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:45.024321 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:45.024283 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q6gs9" event={"ID":"c0625e69-2b51-4675-9080-fc196d8d8070","Type":"ContainerStarted","Data":"69c0fb80e35bb9d365f30aaa259973ddf88420fbec5125a59feb337e1fa4d247"} Jul 18 01:57:45.087634 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:45.087598 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:45.143577 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:45.143537 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:45.291352 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:45.291282 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:45.691827 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:45.691741 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:46.170416 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:46.170384 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:46.222744 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:46.222707 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:46.293953 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:46.293919 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:46.685271 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:46.685171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:47.031243 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:47.031148 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q6gs9" event={"ID":"c0625e69-2b51-4675-9080-fc196d8d8070","Type":"ContainerStarted","Data":"2018e09db5e8d288ff521a9d761f08357053f51cf95df33a43b8ab550f51ee29"} Jul 18 01:57:47.031243 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:47.031210 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q6gs9" event={"ID":"c0625e69-2b51-4675-9080-fc196d8d8070","Type":"ContainerStarted","Data":"4b39bc0cf16a2e24eb70054f389eff37c38ca6b969ab268c892604a714e295a7"} Jul 18 01:57:47.031424 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:47.031285 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:47.092199 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:47.092151 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:47.394844 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:47.394815 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:47.495186 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:47.495148 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:47.887236 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:47.887203 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:48.294583 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:48.294494 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:48.489229 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:48.489194 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:48.688676 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:48.688637 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:49.694241 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:49.694206 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:49.793979 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:49.793934 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:49.855584 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:49.855550 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:49.995140 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:49.995051 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:50.038576 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.038544 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:50.040859 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.040835 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1abbff1-f365-4771-a560-10fee79a4ea2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zjfnx\" (UID: \"d1abbff1-f365-4771-a560-10fee79a4ea2\") " pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:50.048939 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.048919 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zjfnx" Jul 18 01:57:50.095770 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.095737 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:50.193261 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.193209 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q6gs9" podStartSLOduration=20.085952625 podStartE2EDuration="22.193189477s" totalImagesPullingTime="2.107236852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:47.050605625 +0000 UTC m=+54.838942205" watchObservedRunningTime="2026-07-18 01:57:50.193189477 +0000 UTC m=+57.981526076" Jul 18 01:57:50.193755 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.193736 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zjfnx"] Jul 18 01:57:50.197484 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:57:50.197457 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1abbff1_f365_4771_a560_10fee79a4ea2.slice/crio-12ba796d236aef6ed486f68acd14b5b173cf65d8c7a53c50ea18eb17bd2ad724 WatchSource:0}: Error finding container 12ba796d236aef6ed486f68acd14b5b173cf65d8c7a53c50ea18eb17bd2ad724: Status 404 returned error can't find the container with id 12ba796d236aef6ed486f68acd14b5b173cf65d8c7a53c50ea18eb17bd2ad724 Jul 18 01:57:50.293200 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.293105 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:50.394042 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.394010 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:50.461280 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.461249 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:50.520090 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.520059 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:50.685798 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.685740 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:50.786790 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.786739 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:50.838796 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.838755 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:50.967521 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:50.967419 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:51.047033 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:51.046997 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zjfnx" event={"ID":"d1abbff1-f365-4771-a560-10fee79a4ea2","Type":"ContainerStarted","Data":"12ba796d236aef6ed486f68acd14b5b173cf65d8c7a53c50ea18eb17bd2ad724"} Jul 18 01:57:51.069729 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:51.069687 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:51.124727 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:51.124644 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:51.185964 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:51.185934 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:51.381466 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:51.381420 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:51.599046 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:51.598990 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:53.203119 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:53.203077 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:53.302615 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:53.302576 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:57:53.960496 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:53.960469 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dgqnb" Jul 18 01:57:56.064591 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:56.064554 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zjfnx" event={"ID":"d1abbff1-f365-4771-a560-10fee79a4ea2","Type":"ContainerStarted","Data":"56f5aa20cd249236e9086cfdacdec7e45ac70006766649e1454518af851d4ce6"} Jul 18 01:57:57.037451 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:57.037421 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q6gs9" Jul 18 01:57:57.807901 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:57.807854 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkt4b\" (UniqueName: \"kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b\") pod \"network-check-target-m5dx8\" (UID: \"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5\") " pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:57.810630 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:57.810601 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 01:57:57.820504 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:57.820474 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 01:57:57.832111 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:57.832079 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkt4b\" (UniqueName: \"kubernetes.io/projected/009eb0e0-baa3-4a3a-94e3-2ce563f67bd5-kube-api-access-xkt4b\") pod \"network-check-target-m5dx8\" (UID: \"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5\") " pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:57.930363 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:57.930335 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zr2dl\"" Jul 18 01:57:57.938362 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:57.938320 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:57:58.312644 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.312614 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:58.312788 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.312648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:58.315195 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.315170 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 01:57:58.315317 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.315170 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 01:57:58.325136 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.325113 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c41bb40-6562-4a53-8641-8cb1b0a2c4f3-original-pull-secret\") pod \"global-pull-secret-syncer-ljk5t\" (UID: \"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3\") " pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:58.325229 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.325208 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2992267b-15c9-4fa5-be5d-d783f560f9a6-metrics-certs\") pod \"network-metrics-daemon-zjvkm\" (UID: \"2992267b-15c9-4fa5-be5d-d783f560f9a6\") " pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:58.416492 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.416462 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m5dx8"] Jul 18 01:57:58.419924 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:57:58.419896 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod009eb0e0_baa3_4a3a_94e3_2ce563f67bd5.slice/crio-a193f187ab7ed1f1a7e284a534a6b792d68e74e7962dc534a8aa32fd4fc2899b WatchSource:0}: Error finding container a193f187ab7ed1f1a7e284a534a6b792d68e74e7962dc534a8aa32fd4fc2899b: Status 404 returned error can't find the container with id a193f187ab7ed1f1a7e284a534a6b792d68e74e7962dc534a8aa32fd4fc2899b Jul 18 01:57:58.537316 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.537286 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5xbv\"" Jul 18 01:57:58.540241 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.540225 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljk5t" Jul 18 01:57:58.545298 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.545279 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjvkm" Jul 18 01:57:58.662325 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.662294 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ljk5t"] Jul 18 01:57:58.665484 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:57:58.665449 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c41bb40_6562_4a53_8641_8cb1b0a2c4f3.slice/crio-839c30b4eba7e42204da91f7ae76f76d96d771dc61989516ee643ab75e43c457 WatchSource:0}: Error finding container 839c30b4eba7e42204da91f7ae76f76d96d771dc61989516ee643ab75e43c457: Status 404 returned error can't find the container with id 839c30b4eba7e42204da91f7ae76f76d96d771dc61989516ee643ab75e43c457 Jul 18 01:57:58.680990 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:58.680947 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zjvkm"] Jul 18 01:57:58.684199 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:57:58.684175 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2992267b_15c9_4fa5_be5d_d783f560f9a6.slice/crio-151a810bef9d20d17b5d061e1fb96a007e8404b28271a8741350d8f6e6573108 WatchSource:0}: Error finding container 151a810bef9d20d17b5d061e1fb96a007e8404b28271a8741350d8f6e6573108: Status 404 returned error can't find the container with id 151a810bef9d20d17b5d061e1fb96a007e8404b28271a8741350d8f6e6573108 Jul 18 01:57:59.077362 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:59.077320 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ljk5t" event={"ID":"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3","Type":"ContainerStarted","Data":"839c30b4eba7e42204da91f7ae76f76d96d771dc61989516ee643ab75e43c457"} Jul 18 01:57:59.080249 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:59.080200 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zjfnx" event={"ID":"d1abbff1-f365-4771-a560-10fee79a4ea2","Type":"ContainerStarted","Data":"c3ad6586ed0616d5845b393ec786eab45ed7d92bd285ef41aec8538ee3dc3570"} Jul 18 01:57:59.081489 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:59.081445 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m5dx8" event={"ID":"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5","Type":"ContainerStarted","Data":"a193f187ab7ed1f1a7e284a534a6b792d68e74e7962dc534a8aa32fd4fc2899b"} Jul 18 01:57:59.082688 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:59.082658 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjvkm" event={"ID":"2992267b-15c9-4fa5-be5d-d783f560f9a6","Type":"ContainerStarted","Data":"151a810bef9d20d17b5d061e1fb96a007e8404b28271a8741350d8f6e6573108"} Jul 18 01:57:59.925725 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:59.925697 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:57:59.928948 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:57:59.928921 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") pod \"image-registry-67d8b55cdb-26qkq\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:58:00.091118 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.090462 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjvkm" event={"ID":"2992267b-15c9-4fa5-be5d-d783f560f9a6","Type":"ContainerStarted","Data":"ab12df5f4ab4ac31ce193d6f22b21c2362c38fdbbaf2df0185b43b9278d37742"} Jul 18 01:58:00.091118 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.090573 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjvkm" event={"ID":"2992267b-15c9-4fa5-be5d-d783f560f9a6","Type":"ContainerStarted","Data":"0df001d21e3273de44a5287bfc556b166b39f232d794c317a80824748f70ffb3"} Jul 18 01:58:00.107190 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.106526 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zjvkm" podStartSLOduration=66.963725561 podStartE2EDuration="1m8.106507203s" totalImagesPullingTime="1.142781642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:00.10623551 +0000 UTC m=+67.894572096" watchObservedRunningTime="2026-07-18 01:58:00.106507203 +0000 UTC m=+67.894843787" Jul 18 01:58:00.193940 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.193908 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-q7jhh\"" Jul 18 01:58:00.201945 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.201901 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:58:00.362500 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.362421 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67d8b55cdb-26qkq"] Jul 18 01:58:00.366158 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:58:00.366125 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08b361d1_ede4_4a53_9e4c_8e5d58d2b9d1.slice/crio-b097f721463a71713d90a9341c7cdf82ffb3cefb5fd76432e0a73aa099eb6cb0 WatchSource:0}: Error finding container b097f721463a71713d90a9341c7cdf82ffb3cefb5fd76432e0a73aa099eb6cb0: Status 404 returned error can't find the container with id b097f721463a71713d90a9341c7cdf82ffb3cefb5fd76432e0a73aa099eb6cb0 Jul 18 01:58:00.955630 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.955501 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67d8b55cdb-26qkq"] Jul 18 01:58:00.969266 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.968318 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-fqsds"] Jul 18 01:58:00.972481 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.971786 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68ff67f446-8jrs7"] Jul 18 01:58:00.972481 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.971917 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-fqsds" Jul 18 01:58:00.974133 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.974109 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:00.975755 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.975733 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kmfx9\"" Jul 18 01:58:00.976386 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.976091 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 18 01:58:00.976386 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.976298 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 18 01:58:00.977850 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.977652 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-22tsw\"" Jul 18 01:58:00.977850 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.977659 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 18 01:58:00.979411 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.979179 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 18 01:58:00.979553 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.979528 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 18 01:58:00.979990 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.979807 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 18 01:58:00.979990 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:00.979820 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 18 01:58:01.006778 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.006700 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-fqsds"] Jul 18 01:58:01.008087 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.008025 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68ff67f446-8jrs7"] Jul 18 01:58:01.034689 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.034654 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-oauth-serving-cert\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.034854 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.034701 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-config\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.034854 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.034727 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68zqg\" (UniqueName: \"kubernetes.io/projected/6ad60e42-7863-42f9-be18-7f332b9ba8f8-kube-api-access-68zqg\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.034854 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.034762 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-oauth-config\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.034854 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.034793 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn6px\" (UniqueName: \"kubernetes.io/projected/172a9bfd-1995-4d98-847f-a37dc04b2569-kube-api-access-wn6px\") pod \"downloads-5bc67dcd46-fqsds\" (UID: \"172a9bfd-1995-4d98-847f-a37dc04b2569\") " pod="openshift-console/downloads-5bc67dcd46-fqsds" Jul 18 01:58:01.034854 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.034818 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-serving-cert\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.035109 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.034867 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-service-ca\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.070221 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.070184 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c544d7b4f-8b8n7"] Jul 18 01:58:01.072419 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.072393 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.088187 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.088109 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c544d7b4f-8b8n7"] Jul 18 01:58:01.096809 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.096774 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" event={"ID":"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1","Type":"ContainerStarted","Data":"b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812"} Jul 18 01:58:01.097259 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.096834 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" event={"ID":"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1","Type":"ContainerStarted","Data":"b097f721463a71713d90a9341c7cdf82ffb3cefb5fd76432e0a73aa099eb6cb0"} Jul 18 01:58:01.123099 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.123017 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:01.128305 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.128257 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" podStartSLOduration=37.128240545 podStartE2EDuration="37.128240545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:01.127413999 +0000 UTC m=+68.915750573" watchObservedRunningTime="2026-07-18 01:58:01.128240545 +0000 UTC m=+68.916577105" Jul 18 01:58:01.135889 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.135859 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60d356df-e2e2-4536-8c3f-57b66a5bf040-ca-trust-extracted\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.136079 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.135911 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-oauth-serving-cert\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.136079 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.135945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-config\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.136079 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.135987 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68zqg\" (UniqueName: \"kubernetes.io/projected/6ad60e42-7863-42f9-be18-7f332b9ba8f8-kube-api-access-68zqg\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.136079 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136013 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh485\" (UniqueName: \"kubernetes.io/projected/60d356df-e2e2-4536-8c3f-57b66a5bf040-kube-api-access-sh485\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.136079 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136044 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60d356df-e2e2-4536-8c3f-57b66a5bf040-registry-tls\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.136079 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136068 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60d356df-e2e2-4536-8c3f-57b66a5bf040-image-registry-private-configuration\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.136298 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136090 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-oauth-config\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.136298 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136115 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60d356df-e2e2-4536-8c3f-57b66a5bf040-registry-certificates\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.136298 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136149 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn6px\" (UniqueName: \"kubernetes.io/projected/172a9bfd-1995-4d98-847f-a37dc04b2569-kube-api-access-wn6px\") pod \"downloads-5bc67dcd46-fqsds\" (UID: \"172a9bfd-1995-4d98-847f-a37dc04b2569\") " pod="openshift-console/downloads-5bc67dcd46-fqsds" Jul 18 01:58:01.136298 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136173 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-serving-cert\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.136298 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136206 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60d356df-e2e2-4536-8c3f-57b66a5bf040-installation-pull-secrets\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.136298 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136230 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60d356df-e2e2-4536-8c3f-57b66a5bf040-bound-sa-token\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.136298 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136271 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60d356df-e2e2-4536-8c3f-57b66a5bf040-trusted-ca\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.136538 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136306 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-service-ca\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.136801 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136771 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-config\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.136895 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.136868 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-service-ca\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.137438 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.137410 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-oauth-serving-cert\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.139561 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.139535 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-serving-cert\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.139667 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.139589 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-oauth-config\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.146577 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.146543 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn6px\" (UniqueName: \"kubernetes.io/projected/172a9bfd-1995-4d98-847f-a37dc04b2569-kube-api-access-wn6px\") pod \"downloads-5bc67dcd46-fqsds\" (UID: \"172a9bfd-1995-4d98-847f-a37dc04b2569\") " pod="openshift-console/downloads-5bc67dcd46-fqsds" Jul 18 01:58:01.147056 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.147033 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68zqg\" (UniqueName: \"kubernetes.io/projected/6ad60e42-7863-42f9-be18-7f332b9ba8f8-kube-api-access-68zqg\") pod \"console-68ff67f446-8jrs7\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.213456 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.213364 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:01.237404 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.237361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60d356df-e2e2-4536-8c3f-57b66a5bf040-ca-trust-extracted\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.237585 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.237561 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh485\" (UniqueName: \"kubernetes.io/projected/60d356df-e2e2-4536-8c3f-57b66a5bf040-kube-api-access-sh485\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.237653 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.237608 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60d356df-e2e2-4536-8c3f-57b66a5bf040-registry-tls\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.237711 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.237648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60d356df-e2e2-4536-8c3f-57b66a5bf040-image-registry-private-configuration\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.237711 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.237679 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60d356df-e2e2-4536-8c3f-57b66a5bf040-registry-certificates\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.237815 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.237753 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60d356df-e2e2-4536-8c3f-57b66a5bf040-installation-pull-secrets\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.237815 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.237768 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60d356df-e2e2-4536-8c3f-57b66a5bf040-ca-trust-extracted\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.237815 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.237778 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60d356df-e2e2-4536-8c3f-57b66a5bf040-bound-sa-token\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.237955 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.237849 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60d356df-e2e2-4536-8c3f-57b66a5bf040-trusted-ca\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.238808 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.238778 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60d356df-e2e2-4536-8c3f-57b66a5bf040-registry-certificates\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.238926 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.238814 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60d356df-e2e2-4536-8c3f-57b66a5bf040-trusted-ca\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.240636 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.240595 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60d356df-e2e2-4536-8c3f-57b66a5bf040-registry-tls\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.240636 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.240629 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60d356df-e2e2-4536-8c3f-57b66a5bf040-installation-pull-secrets\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.241409 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.241388 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60d356df-e2e2-4536-8c3f-57b66a5bf040-image-registry-private-configuration\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.247215 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.247164 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60d356df-e2e2-4536-8c3f-57b66a5bf040-bound-sa-token\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.248110 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.248086 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh485\" (UniqueName: \"kubernetes.io/projected/60d356df-e2e2-4536-8c3f-57b66a5bf040-kube-api-access-sh485\") pod \"image-registry-6c544d7b4f-8b8n7\" (UID: \"60d356df-e2e2-4536-8c3f-57b66a5bf040\") " pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.286062 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.286022 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-fqsds" Jul 18 01:58:01.293820 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.293792 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:01.312531 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.312492 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:01.384427 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.384392 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:01.410708 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.410673 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:01.925125 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.925061 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:01.987648 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:01.987561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:02.104164 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:02.101835 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:58:04.198003 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:04.197950 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:04.296915 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:04.296881 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:04.811160 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:04.811106 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:04.906828 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:04.906797 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:04.976055 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:04.976015 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:05.068220 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:05.068133 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:05.658591 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:05.658557 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:05.710451 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:05.710424 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-fqsds"] Jul 18 01:58:05.713079 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:58:05.713046 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod172a9bfd_1995_4d98_847f_a37dc04b2569.slice/crio-56134d85a15b29726d0a585faef0f0b549861ba7ba2368ef8881a64a6c3239f6 WatchSource:0}: Error finding container 56134d85a15b29726d0a585faef0f0b549861ba7ba2368ef8881a64a6c3239f6: Status 404 returned error can't find the container with id 56134d85a15b29726d0a585faef0f0b549861ba7ba2368ef8881a64a6c3239f6 Jul 18 01:58:05.757670 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:05.757636 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:05.935789 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:05.935713 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68ff67f446-8jrs7"] Jul 18 01:58:05.938086 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:58:05.938049 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad60e42_7863_42f9_be18_7f332b9ba8f8.slice/crio-84920301ccddc642324b8bb966e41e3f5bbecb8f9c61558aa21b93b57508eecf WatchSource:0}: Error finding container 84920301ccddc642324b8bb966e41e3f5bbecb8f9c61558aa21b93b57508eecf: Status 404 returned error can't find the container with id 84920301ccddc642324b8bb966e41e3f5bbecb8f9c61558aa21b93b57508eecf Jul 18 01:58:05.938537 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:05.938511 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c544d7b4f-8b8n7"] Jul 18 01:58:05.942351 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:58:05.942330 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d356df_e2e2_4536_8c3f_57b66a5bf040.slice/crio-c3af975a7db1c8bd6d11dea4f9de69be4652d7cd5446c1e025ae68a6fa616b5a WatchSource:0}: Error finding container c3af975a7db1c8bd6d11dea4f9de69be4652d7cd5446c1e025ae68a6fa616b5a: Status 404 returned error can't find the container with id c3af975a7db1c8bd6d11dea4f9de69be4652d7cd5446c1e025ae68a6fa616b5a Jul 18 01:58:06.056057 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.056029 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:06.116063 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.116022 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ljk5t" event={"ID":"5c41bb40-6562-4a53-8641-8cb1b0a2c4f3","Type":"ContainerStarted","Data":"b0d31259e7bbc74d9640b5613f32c3108c98dd8289b143b6031bfeb47b7fde46"} Jul 18 01:58:06.117232 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.117208 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-fqsds" event={"ID":"172a9bfd-1995-4d98-847f-a37dc04b2569","Type":"ContainerStarted","Data":"56134d85a15b29726d0a585faef0f0b549861ba7ba2368ef8881a64a6c3239f6"} Jul 18 01:58:06.119140 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.119114 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zjfnx" event={"ID":"d1abbff1-f365-4771-a560-10fee79a4ea2","Type":"ContainerStarted","Data":"6d99f453504f18cb03d64592cdafce0f4acd8363f8a105f6bc7b376e88027363"} Jul 18 01:58:06.120735 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.120715 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m5dx8" event={"ID":"009eb0e0-baa3-4a3a-94e3-2ce563f67bd5","Type":"ContainerStarted","Data":"2fefb9ee7551c87701e0291807e4606515940aad5d7ae25af3f3cf694369509c"} Jul 18 01:58:06.120836 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.120766 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:58:06.122248 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.122223 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" event={"ID":"60d356df-e2e2-4536-8c3f-57b66a5bf040","Type":"ContainerStarted","Data":"b6d12c6fba70b82e532342c907627850c2c2c216c2763a60c328de0328f006b1"} Jul 18 01:58:06.122345 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.122263 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" event={"ID":"60d356df-e2e2-4536-8c3f-57b66a5bf040","Type":"ContainerStarted","Data":"c3af975a7db1c8bd6d11dea4f9de69be4652d7cd5446c1e025ae68a6fa616b5a"} Jul 18 01:58:06.122345 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.122315 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:06.123456 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.123439 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ff67f446-8jrs7" event={"ID":"6ad60e42-7863-42f9-be18-7f332b9ba8f8","Type":"ContainerStarted","Data":"84920301ccddc642324b8bb966e41e3f5bbecb8f9c61558aa21b93b57508eecf"} Jul 18 01:58:06.131893 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.131848 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ljk5t" podStartSLOduration=67.229139248 podStartE2EDuration="1m14.131833775s" totalImagesPullingTime="6.902694527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:06.130633479 +0000 UTC m=+73.918970062" watchObservedRunningTime="2026-07-18 01:58:06.131833775 +0000 UTC m=+73.920170339" Jul 18 01:58:06.135653 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.135628 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:06.148460 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.148409 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zjfnx" podStartSLOduration=16.899715395 podStartE2EDuration="32.148394304s" totalImagesPullingTime="15.248678909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:06.147658963 +0000 UTC m=+73.935995545" watchObservedRunningTime="2026-07-18 01:58:06.148394304 +0000 UTC m=+73.936730885" Jul 18 01:58:06.165798 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.165740 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" podStartSLOduration=5.1657279 podStartE2EDuration="5.1657279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:06.165039758 +0000 UTC m=+73.953376340" watchObservedRunningTime="2026-07-18 01:58:06.1657279 +0000 UTC m=+73.954064476" Jul 18 01:58:06.180043 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:06.179987 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m5dx8" podStartSLOduration=67.047693785 podStartE2EDuration="1m14.179954737s" totalImagesPullingTime="7.132260952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:06.179218622 +0000 UTC m=+73.967555206" watchObservedRunningTime="2026-07-18 01:58:06.179954737 +0000 UTC m=+73.968291319" Jul 18 01:58:07.521515 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:07.521447 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:07.622411 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:07.622313 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:07.797946 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:07.797870 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:07.862945 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:07.862847 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:09.004002 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:09.003873 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:09.104844 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:09.104785 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:09.138102 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:09.137954 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ff67f446-8jrs7" event={"ID":"6ad60e42-7863-42f9-be18-7f332b9ba8f8","Type":"ContainerStarted","Data":"b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9"} Jul 18 01:58:09.155434 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:09.155376 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68ff67f446-8jrs7" podStartSLOduration=6.113510402 podStartE2EDuration="9.155355329s" totalImagesPullingTime="3.041844927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:09.153928271 +0000 UTC m=+76.942264854" watchObservedRunningTime="2026-07-18 01:58:09.155355329 +0000 UTC m=+76.943691914" Jul 18 01:58:09.602675 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:09.602630 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:09.660362 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:09.660326 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:11.000367 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:11.000322 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:11.087980 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:11.087939 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:11.238386 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:11.238350 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:11.294619 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:11.294545 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:11.294619 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:11.294611 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:11.300440 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:11.300409 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:12.154502 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:12.154472 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:58:12.798639 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:12.798603 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:14.600745 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:14.600696 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:14.701653 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:14.701617 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:16.405624 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:16.405489 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:16.868317 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:16.868231 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:16.968635 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:16.968597 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:17.688223 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:17.688185 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:17.788995 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:17.788930 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:20.078284 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:20.078254 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:20.721131 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:20.721091 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:20.777776 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:20.777702 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:20.831214 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:20.831180 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:21.578652 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:21.578611 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:21.679922 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:21.679882 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:22.109001 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.108846 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:58:22.123090 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.123040 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:22.190540 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.190458 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-fqsds" event={"ID":"172a9bfd-1995-4d98-847f-a37dc04b2569","Type":"ContainerStarted","Data":"da23edd0fbb3827347ad8338928b4c0529180db1f5391d1b81a2d312038d9b92"} Jul 18 01:58:22.190858 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.190832 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-fqsds" Jul 18 01:58:22.199434 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.196529 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-fqsds" Jul 18 01:58:22.208433 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.208372 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-fqsds" podStartSLOduration=6.007088942 podStartE2EDuration="22.208353136s" totalImagesPullingTime="16.201264194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:22.206815291 +0000 UTC m=+89.995151886" watchObservedRunningTime="2026-07-18 01:58:22.208353136 +0000 UTC m=+89.996689718" Jul 18 01:58:22.368675 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.368632 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:22.402211 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.402172 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ws4ng"] Jul 18 01:58:22.408598 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.408561 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.413648 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.412839 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 18 01:58:22.413648 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.412902 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s9bfv\"" Jul 18 01:58:22.413648 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.413158 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 18 01:58:22.413648 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.413200 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 18 01:58:22.413648 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.413337 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 18 01:58:22.413648 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.413395 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 18 01:58:22.413648 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.413527 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 18 01:58:22.457743 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.457666 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:22.523155 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.522640 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-wtmp\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.523155 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.522713 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-textfile\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.523155 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.522749 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.523155 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.522779 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtp69\" (UniqueName: \"kubernetes.io/projected/412b7285-a7b6-4388-85e2-9c2c337245e4-kube-api-access-vtp69\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.523155 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.522811 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/412b7285-a7b6-4388-85e2-9c2c337245e4-sys\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.523155 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.522840 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/412b7285-a7b6-4388-85e2-9c2c337245e4-metrics-client-ca\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.523155 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.522872 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/412b7285-a7b6-4388-85e2-9c2c337245e4-root\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.523155 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.522904 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-tls\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.523155 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.523004 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.623891 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.623852 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.624463 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.624435 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-wtmp\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.624597 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.624489 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-textfile\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.624597 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.624519 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.624597 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.624545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtp69\" (UniqueName: \"kubernetes.io/projected/412b7285-a7b6-4388-85e2-9c2c337245e4-kube-api-access-vtp69\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.624597 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.624579 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/412b7285-a7b6-4388-85e2-9c2c337245e4-sys\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.624824 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.624647 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-wtmp\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.624824 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.624783 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/412b7285-a7b6-4388-85e2-9c2c337245e4-sys\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.624933 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.624857 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/412b7285-a7b6-4388-85e2-9c2c337245e4-metrics-client-ca\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.625160 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.625131 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/412b7285-a7b6-4388-85e2-9c2c337245e4-root\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.625667 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.625179 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-tls\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.625667 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.625185 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.625667 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.625254 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/412b7285-a7b6-4388-85e2-9c2c337245e4-root\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.625667 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:58:22.625291 2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 01:58:22.625667 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:58:22.625362 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-tls podName:412b7285-a7b6-4388-85e2-9c2c337245e4 nodeName:}" failed. No retries permitted until 2026-07-18 01:58:23.125340663 +0000 UTC m=+90.913677224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-tls") pod "node-exporter-ws4ng" (UID: "412b7285-a7b6-4388-85e2-9c2c337245e4") : secret "node-exporter-tls" not found Jul 18 01:58:22.625667 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.625389 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-textfile\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.625938 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.625736 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/412b7285-a7b6-4388-85e2-9c2c337245e4-metrics-client-ca\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.627053 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.626991 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.627172 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.627148 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:22.635769 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.635741 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtp69\" (UniqueName: \"kubernetes.io/projected/412b7285-a7b6-4388-85e2-9c2c337245e4-kube-api-access-vtp69\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:22.727563 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.727486 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:22.779695 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.779657 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:22.857844 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:22.857807 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:23.130281 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:23.130239 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-tls\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:23.130564 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:58:23.130396 2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 01:58:23.130564 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:58:23.130487 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-tls podName:412b7285-a7b6-4388-85e2-9c2c337245e4 nodeName:}" failed. No retries permitted until 2026-07-18 01:58:24.13046478 +0000 UTC m=+91.918801361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-tls") pod "node-exporter-ws4ng" (UID: "412b7285-a7b6-4388-85e2-9c2c337245e4") : secret "node-exporter-tls" not found Jul 18 01:58:23.200626 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:23.200593 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:23.368549 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:23.368515 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:23.443230 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:23.443132 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:23.591335 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:23.591297 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:24.140660 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:24.140628 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-tls\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:24.143307 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:24.143279 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/412b7285-a7b6-4388-85e2-9c2c337245e4-node-exporter-tls\") pod \"node-exporter-ws4ng\" (UID: \"412b7285-a7b6-4388-85e2-9c2c337245e4\") " pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:24.222830 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:24.222787 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ws4ng" Jul 18 01:58:24.258112 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:58:24.258080 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod412b7285_a7b6_4388_85e2_9c2c337245e4.slice/crio-6a262825eb3c7c08f4bbae2d1d2cbd8de22d400b0798c9e39bc2daf4a99ef8cb WatchSource:0}: Error finding container 6a262825eb3c7c08f4bbae2d1d2cbd8de22d400b0798c9e39bc2daf4a99ef8cb: Status 404 returned error can't find the container with id 6a262825eb3c7c08f4bbae2d1d2cbd8de22d400b0798c9e39bc2daf4a99ef8cb Jul 18 01:58:25.203328 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:25.203286 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ws4ng" event={"ID":"412b7285-a7b6-4388-85e2-9c2c337245e4","Type":"ContainerStarted","Data":"6a262825eb3c7c08f4bbae2d1d2cbd8de22d400b0798c9e39bc2daf4a99ef8cb"} Jul 18 01:58:25.311214 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:25.311176 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:25.383833 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:25.383791 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:25.485009 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:25.484910 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:25.541465 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:25.541425 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:26.209228 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:26.209185 2573 generic.go:356] "Generic (PLEG): container finished" podID="412b7285-a7b6-4388-85e2-9c2c337245e4" containerID="726e129b97c46536a1af14f8438d05dbc9328c3c06c9824ac5d00803bed16427" exitCode=0 Jul 18 01:58:26.209760 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:26.209258 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ws4ng" event={"ID":"412b7285-a7b6-4388-85e2-9c2c337245e4","Type":"ContainerDied","Data":"726e129b97c46536a1af14f8438d05dbc9328c3c06c9824ac5d00803bed16427"} Jul 18 01:58:27.125304 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.125257 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" podUID="08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" containerName="registry" containerID="cri-o://b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812" gracePeriod=30 Jul 18 01:58:27.134481 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.134459 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c544d7b4f-8b8n7" Jul 18 01:58:27.151646 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.151616 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:27.216036 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.215990 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ws4ng" event={"ID":"412b7285-a7b6-4388-85e2-9c2c337245e4","Type":"ContainerStarted","Data":"1062826bbfc70420b38891f0a3092187cd02bfae6e9a614738e012f2728ae4be"} Jul 18 01:58:27.216457 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.216063 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ws4ng" event={"ID":"412b7285-a7b6-4388-85e2-9c2c337245e4","Type":"ContainerStarted","Data":"3d2efd5e73744af6e99c854ed2202fecc63a7a1dc2fbf65e9193d8fc36410d61"} Jul 18 01:58:27.255762 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.255541 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ws4ng" podStartSLOduration=3.782042085 podStartE2EDuration="5.255517741s" totalImagesPullingTime="1.371928397s" totalInitContainerRuntime="101.547259ms" isStatefulPod=true podCreationTimestamp="2026-07-18 01:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:27.253875175 +0000 UTC m=+95.042211757" watchObservedRunningTime="2026-07-18 01:58:27.255517741 +0000 UTC m=+95.043854324" Jul 18 01:58:27.282793 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.282729 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:27.376446 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.376328 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:27.435315 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.435121 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:58:27.459729 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.459659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:27.470904 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.470873 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-bound-sa-token\") pod \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " Jul 18 01:58:27.471094 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.470921 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-ca-trust-extracted\") pod \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " Jul 18 01:58:27.471094 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.470949 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bj8fd\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-kube-api-access-bj8fd\") pod \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " Jul 18 01:58:27.471094 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.471002 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") pod \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " Jul 18 01:58:27.471094 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.471044 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-trusted-ca\") pod \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " Jul 18 01:58:27.471094 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.471071 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-installation-pull-secrets\") pod \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " Jul 18 01:58:27.471094 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.471092 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-certificates\") pod \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " Jul 18 01:58:27.471373 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.471126 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-image-registry-private-configuration\") pod \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\" (UID: \"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1\") " Jul 18 01:58:27.476594 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.476312 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:58:27.476737 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.476632 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:58:27.480566 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.480525 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:58:27.480981 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.480930 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:58:27.481073 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.481030 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:58:27.481164 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.481129 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-kube-api-access-bj8fd" (OuterVolumeSpecName: "kube-api-access-bj8fd") pod "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1"). InnerVolumeSpecName "kube-api-access-bj8fd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:58:27.481549 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.481518 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:58:27.482791 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.482760 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" (UID: "08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:58:27.571677 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.571637 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-bound-sa-token\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:58:27.571677 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.571679 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-ca-trust-extracted\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:58:27.571896 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.571694 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bj8fd\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-kube-api-access-bj8fd\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:58:27.571896 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.571708 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-tls\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:58:27.571896 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.571716 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-trusted-ca\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:58:27.571896 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.571729 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-installation-pull-secrets\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:58:27.571896 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.571741 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-registry-certificates\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:58:27.571896 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.571750 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1-image-registry-private-configuration\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:58:27.937617 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:27.937571 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:28.116748 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.116711 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:28.217840 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.217758 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:28.221210 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.221179 2573 generic.go:356] "Generic (PLEG): container finished" podID="08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" containerID="b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812" exitCode=0 Jul 18 01:58:28.221343 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.221225 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" Jul 18 01:58:28.221343 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.221270 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" event={"ID":"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1","Type":"ContainerDied","Data":"b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812"} Jul 18 01:58:28.221343 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.221327 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67d8b55cdb-26qkq" event={"ID":"08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1","Type":"ContainerDied","Data":"b097f721463a71713d90a9341c7cdf82ffb3cefb5fd76432e0a73aa099eb6cb0"} Jul 18 01:58:28.221490 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.221354 2573 scope.go:117] "RemoveContainer" containerID="b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812" Jul 18 01:58:28.231199 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.231173 2573 scope.go:117] "RemoveContainer" containerID="b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812" Jul 18 01:58:28.231556 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:58:28.231526 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812\": container with ID starting with b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812 not found: ID does not exist" containerID="b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812" Jul 18 01:58:28.231651 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.231562 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812"} err="failed to get container status \"b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812\": rpc error: code = NotFound desc = could not find container \"b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812\": container with ID starting with b61b4b5ad758bab268528ba8509c9bb9958fe80e24d364d614aa38329fcd3812 not found: ID does not exist" Jul 18 01:58:28.243709 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.243677 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67d8b55cdb-26qkq"] Jul 18 01:58:28.246920 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.246892 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67d8b55cdb-26qkq"] Jul 18 01:58:28.571686 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.571608 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:28.722393 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.722358 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" path="/var/lib/kubelet/pods/08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1/volumes" Jul 18 01:58:28.817684 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.817637 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:28.917258 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:28.917223 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:30.582158 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:30.582118 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:30.635240 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:30.635205 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:36.056137 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:36.056100 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:36.110990 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:36.110941 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:37.131110 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:37.131077 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m5dx8" Jul 18 01:58:37.718422 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:37.718383 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68ff67f446-8jrs7"] Jul 18 01:58:37.820894 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:37.820852 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:37.874770 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:37.874740 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:37.928414 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:37.928373 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:49.112135 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:49.112101 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j9bdq_25808b10-82d5-4db1-867a-7791b405f163/serve-healthcheck-canary/0.log" Jul 18 01:58:51.055517 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:51.055484 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:51.111194 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:51.111159 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:58.075261 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:58.075224 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:58.176700 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:58.176653 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:58:58.276807 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:58:58.276761 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:02.738612 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:02.738544 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68ff67f446-8jrs7" podUID="6ad60e42-7863-42f9-be18-7f332b9ba8f8" containerName="console" containerID="cri-o://b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9" gracePeriod=15 Jul 18 01:59:03.018198 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.018163 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:59:03.138527 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.138493 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-oauth-config\") pod \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " Jul 18 01:59:03.138720 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.138558 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-config\") pod \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " Jul 18 01:59:03.138720 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.138586 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-service-ca\") pod \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " Jul 18 01:59:03.138720 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.138605 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-oauth-serving-cert\") pod \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " Jul 18 01:59:03.138720 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.138631 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-serving-cert\") pod \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " Jul 18 01:59:03.138720 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.138660 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68zqg\" (UniqueName: \"kubernetes.io/projected/6ad60e42-7863-42f9-be18-7f332b9ba8f8-kube-api-access-68zqg\") pod \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\" (UID: \"6ad60e42-7863-42f9-be18-7f332b9ba8f8\") " Jul 18 01:59:03.139001 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.138951 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6ad60e42-7863-42f9-be18-7f332b9ba8f8" (UID: "6ad60e42-7863-42f9-be18-7f332b9ba8f8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:59:03.139072 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.139008 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-config" (OuterVolumeSpecName: "console-config") pod "6ad60e42-7863-42f9-be18-7f332b9ba8f8" (UID: "6ad60e42-7863-42f9-be18-7f332b9ba8f8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:59:03.139072 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.139022 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-service-ca" (OuterVolumeSpecName: "service-ca") pod "6ad60e42-7863-42f9-be18-7f332b9ba8f8" (UID: "6ad60e42-7863-42f9-be18-7f332b9ba8f8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:59:03.140910 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.140887 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6ad60e42-7863-42f9-be18-7f332b9ba8f8" (UID: "6ad60e42-7863-42f9-be18-7f332b9ba8f8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:59:03.141040 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.140917 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad60e42-7863-42f9-be18-7f332b9ba8f8-kube-api-access-68zqg" (OuterVolumeSpecName: "kube-api-access-68zqg") pod "6ad60e42-7863-42f9-be18-7f332b9ba8f8" (UID: "6ad60e42-7863-42f9-be18-7f332b9ba8f8"). InnerVolumeSpecName "kube-api-access-68zqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:59:03.141040 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.140934 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6ad60e42-7863-42f9-be18-7f332b9ba8f8" (UID: "6ad60e42-7863-42f9-be18-7f332b9ba8f8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:59:03.239946 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.239859 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-serving-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:59:03.239946 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.239892 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68zqg\" (UniqueName: \"kubernetes.io/projected/6ad60e42-7863-42f9-be18-7f332b9ba8f8-kube-api-access-68zqg\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:59:03.239946 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.239902 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-oauth-config\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:59:03.239946 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.239912 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-console-config\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:59:03.239946 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.239920 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-service-ca\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:59:03.239946 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.239928 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ad60e42-7863-42f9-be18-7f332b9ba8f8-oauth-serving-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 01:59:03.341807 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.341768 2573 generic.go:356] "Generic (PLEG): container finished" podID="6ad60e42-7863-42f9-be18-7f332b9ba8f8" containerID="b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9" exitCode=0 Jul 18 01:59:03.341995 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.341818 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ff67f446-8jrs7" Jul 18 01:59:03.341995 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.341846 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ff67f446-8jrs7" event={"ID":"6ad60e42-7863-42f9-be18-7f332b9ba8f8","Type":"ContainerDied","Data":"b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9"} Jul 18 01:59:03.341995 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.341895 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ff67f446-8jrs7" event={"ID":"6ad60e42-7863-42f9-be18-7f332b9ba8f8","Type":"ContainerDied","Data":"84920301ccddc642324b8bb966e41e3f5bbecb8f9c61558aa21b93b57508eecf"} Jul 18 01:59:03.341995 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.341923 2573 scope.go:117] "RemoveContainer" containerID="b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9" Jul 18 01:59:03.350198 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.350176 2573 scope.go:117] "RemoveContainer" containerID="b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9" Jul 18 01:59:03.350475 ip-10-0-130-187 kubenswrapper[2573]: E0718 01:59:03.350457 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9\": container with ID starting with b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9 not found: ID does not exist" containerID="b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9" Jul 18 01:59:03.350526 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.350486 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9"} err="failed to get container status \"b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9\": rpc error: code = NotFound desc = could not find container \"b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9\": container with ID starting with b089ced2a6eb638c2aad74b9b131ed80588daa726e1bbe394403f6e6f1ff75a9 not found: ID does not exist" Jul 18 01:59:03.362324 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.362294 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68ff67f446-8jrs7"] Jul 18 01:59:03.365718 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:03.365693 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68ff67f446-8jrs7"] Jul 18 01:59:04.721758 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:04.721723 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ad60e42-7863-42f9-be18-7f332b9ba8f8" path="/var/lib/kubelet/pods/6ad60e42-7863-42f9-be18-7f332b9ba8f8/volumes" Jul 18 01:59:19.485393 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:19.485358 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:33.824870 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:33.824828 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:41.573321 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:41.573224 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:41.772993 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:41.772938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:52.038797 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.038712 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b75964cff-njh2t"] Jul 18 01:59:52.039197 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.039042 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad60e42-7863-42f9-be18-7f332b9ba8f8" containerName="console" Jul 18 01:59:52.039197 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.039056 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad60e42-7863-42f9-be18-7f332b9ba8f8" containerName="console" Jul 18 01:59:52.039197 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.039079 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" containerName="registry" Jul 18 01:59:52.039197 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.039085 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" containerName="registry" Jul 18 01:59:52.039197 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.039122 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b361d1-ede4-4a53-9e4c-8e5d58d2b9d1" containerName="registry" Jul 18 01:59:52.039197 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.039129 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad60e42-7863-42f9-be18-7f332b9ba8f8" containerName="console" Jul 18 01:59:52.040870 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.040855 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.043292 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.043268 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 18 01:59:52.043406 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.043317 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-22tsw\"" Jul 18 01:59:52.044267 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.044253 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 18 01:59:52.044352 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.044337 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 18 01:59:52.044437 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.044418 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 18 01:59:52.044519 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.044501 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 18 01:59:52.049122 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.049102 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 18 01:59:52.053710 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.053686 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b75964cff-njh2t"] Jul 18 01:59:52.096908 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.096867 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:52.176546 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.176509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-oauth-config\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.176546 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.176548 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-console-config\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.176731 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.176571 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-serving-cert\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.176731 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.176620 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-service-ca\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.176731 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.176685 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-oauth-serving-cert\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.176731 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.176706 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpnvh\" (UniqueName: \"kubernetes.io/projected/fdef514d-2dff-445a-8952-9878f932f251-kube-api-access-dpnvh\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.176731 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.176732 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-trusted-ca-bundle\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.197953 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.197903 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:52.277789 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.277754 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-trusted-ca-bundle\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.277789 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.277798 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-oauth-config\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.278069 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.277815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-console-config\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.278069 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.277835 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-serving-cert\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.278069 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.277897 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-service-ca\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.278069 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.277961 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-oauth-serving-cert\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.278069 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.278015 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpnvh\" (UniqueName: \"kubernetes.io/projected/fdef514d-2dff-445a-8952-9878f932f251-kube-api-access-dpnvh\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.278707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.278680 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-service-ca\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.278707 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.278695 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-oauth-serving-cert\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.278837 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.278823 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-console-config\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.278900 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.278878 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-trusted-ca-bundle\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.280301 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.280284 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-oauth-config\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.280426 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.280409 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-serving-cert\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.285433 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.285412 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpnvh\" (UniqueName: \"kubernetes.io/projected/fdef514d-2dff-445a-8952-9878f932f251-kube-api-access-dpnvh\") pod \"console-5b75964cff-njh2t\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.298154 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.298096 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:52.349508 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.349476 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:52.351363 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.351334 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b75964cff-njh2t" Jul 18 01:59:52.473265 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.473227 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b75964cff-njh2t"] Jul 18 01:59:52.476573 ip-10-0-130-187 kubenswrapper[2573]: W0718 01:59:52.476546 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdef514d_2dff_445a_8952_9878f932f251.slice/crio-5d1e409be16ef71ec0d956e1d26b7a32c628467fdfa08add2f3965743e81c6fd WatchSource:0}: Error finding container 5d1e409be16ef71ec0d956e1d26b7a32c628467fdfa08add2f3965743e81c6fd: Status 404 returned error can't find the container with id 5d1e409be16ef71ec0d956e1d26b7a32c628467fdfa08add2f3965743e81c6fd Jul 18 01:59:52.499191 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.499165 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b75964cff-njh2t" event={"ID":"fdef514d-2dff-445a-8952-9878f932f251","Type":"ContainerStarted","Data":"5d1e409be16ef71ec0d956e1d26b7a32c628467fdfa08add2f3965743e81c6fd"} Jul 18 01:59:52.834339 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:52.834259 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:53.503940 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:53.503905 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b75964cff-njh2t" event={"ID":"fdef514d-2dff-445a-8952-9878f932f251","Type":"ContainerStarted","Data":"34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3"} Jul 18 01:59:53.521790 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:53.521739 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b75964cff-njh2t" podStartSLOduration=1.521721768 podStartE2EDuration="1.521721768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:59:53.520537427 +0000 UTC m=+181.308874012" watchObservedRunningTime="2026-07-18 01:59:53.521721768 +0000 UTC m=+181.310058425" Jul 18 01:59:56.771191 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:56.771151 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 01:59:56.827521 ip-10-0-130-187 kubenswrapper[2573]: I0718 01:59:56.827483 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:02.351813 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.351772 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b75964cff-njh2t" Jul 18 02:00:02.352233 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.351846 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b75964cff-njh2t" Jul 18 02:00:02.356401 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.356379 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b75964cff-njh2t" Jul 18 02:00:02.539157 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.539128 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b75964cff-njh2t" Jul 18 02:00:02.645484 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.645450 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8669ff4d45-chxdr"] Jul 18 02:00:02.648071 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.648053 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.659642 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.659616 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8669ff4d45-chxdr"] Jul 18 02:00:02.747864 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.747826 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmgwd\" (UniqueName: \"kubernetes.io/projected/69c85dc1-0740-4344-9f15-518f1210a748-kube-api-access-dmgwd\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.747864 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.747864 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-console-config\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.748108 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.747886 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-trusted-ca-bundle\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.748108 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.747935 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-oauth-serving-cert\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.748108 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.748009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69c85dc1-0740-4344-9f15-518f1210a748-console-oauth-config\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.748108 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.748066 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69c85dc1-0740-4344-9f15-518f1210a748-console-serving-cert\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.748108 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.748083 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-service-ca\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.849052 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.849016 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69c85dc1-0740-4344-9f15-518f1210a748-console-serving-cert\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.849052 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.849053 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-service-ca\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.849278 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.849078 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmgwd\" (UniqueName: \"kubernetes.io/projected/69c85dc1-0740-4344-9f15-518f1210a748-kube-api-access-dmgwd\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.849278 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.849117 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-console-config\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.849355 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.849282 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-trusted-ca-bundle\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.849355 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.849317 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-oauth-serving-cert\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.849458 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.849358 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69c85dc1-0740-4344-9f15-518f1210a748-console-oauth-config\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.849907 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.849877 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-service-ca\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.850051 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.849877 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-console-config\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.850051 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.849933 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-oauth-serving-cert\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.850359 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.850333 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69c85dc1-0740-4344-9f15-518f1210a748-trusted-ca-bundle\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.851690 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.851673 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69c85dc1-0740-4344-9f15-518f1210a748-console-serving-cert\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.851739 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.851696 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69c85dc1-0740-4344-9f15-518f1210a748-console-oauth-config\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.857036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.857016 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmgwd\" (UniqueName: \"kubernetes.io/projected/69c85dc1-0740-4344-9f15-518f1210a748-kube-api-access-dmgwd\") pod \"console-8669ff4d45-chxdr\" (UID: \"69c85dc1-0740-4344-9f15-518f1210a748\") " pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:02.957160 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:02.957060 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:03.033735 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:03.033702 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:03.086381 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:03.086347 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8669ff4d45-chxdr"] Jul 18 02:00:03.089947 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:00:03.089917 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69c85dc1_0740_4344_9f15_518f1210a748.slice/crio-4ab796f0b3e69aa79848ea8f8449fd68f777a864ec5ba787a5d5d7e62e5d2fb7 WatchSource:0}: Error finding container 4ab796f0b3e69aa79848ea8f8449fd68f777a864ec5ba787a5d5d7e62e5d2fb7: Status 404 returned error can't find the container with id 4ab796f0b3e69aa79848ea8f8449fd68f777a864ec5ba787a5d5d7e62e5d2fb7 Jul 18 02:00:03.099460 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:03.099422 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:03.540363 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:03.540330 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8669ff4d45-chxdr" event={"ID":"69c85dc1-0740-4344-9f15-518f1210a748","Type":"ContainerStarted","Data":"61690e3d77bd128aeaf490454bee4eacde6b9c0c554bde51a59e62ef37433e8e"} Jul 18 02:00:03.540741 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:03.540381 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8669ff4d45-chxdr" event={"ID":"69c85dc1-0740-4344-9f15-518f1210a748","Type":"ContainerStarted","Data":"4ab796f0b3e69aa79848ea8f8449fd68f777a864ec5ba787a5d5d7e62e5d2fb7"} Jul 18 02:00:03.557143 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:03.557086 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8669ff4d45-chxdr" podStartSLOduration=1.55706611 podStartE2EDuration="1.55706611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:00:03.556041024 +0000 UTC m=+191.344377608" watchObservedRunningTime="2026-07-18 02:00:03.55706611 +0000 UTC m=+191.345402692" Jul 18 02:00:03.748033 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:03.747989 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:08.748397 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:08.748364 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:08.804334 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:08.804301 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:12.957711 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:12.957677 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:12.958189 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:12.957738 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:12.962157 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:12.962135 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:13.576620 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:13.576589 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8669ff4d45-chxdr" Jul 18 02:00:13.619859 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:13.619826 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b75964cff-njh2t"] Jul 18 02:00:13.721683 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:13.721651 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:13.822119 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:13.822088 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:13.983164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:13.983125 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:19.368344 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:19.368312 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:19.423296 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:19.423264 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:21.068029 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:21.067995 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:21.169363 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:21.169329 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:22.479727 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:22.479688 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:22.577691 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:22.577659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:38.644291 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.644243 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b75964cff-njh2t" podUID="fdef514d-2dff-445a-8952-9878f932f251" containerName="console" containerID="cri-o://34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3" gracePeriod=15 Jul 18 02:00:38.886304 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.886278 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b75964cff-njh2t" Jul 18 02:00:38.996939 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.996845 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-serving-cert\") pod \"fdef514d-2dff-445a-8952-9878f932f251\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " Jul 18 02:00:38.996939 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.996881 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-oauth-serving-cert\") pod \"fdef514d-2dff-445a-8952-9878f932f251\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " Jul 18 02:00:38.996939 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.996899 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-service-ca\") pod \"fdef514d-2dff-445a-8952-9878f932f251\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " Jul 18 02:00:38.996939 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.996926 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-trusted-ca-bundle\") pod \"fdef514d-2dff-445a-8952-9878f932f251\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " Jul 18 02:00:38.997303 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.996957 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-oauth-config\") pod \"fdef514d-2dff-445a-8952-9878f932f251\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " Jul 18 02:00:38.997303 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.997001 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpnvh\" (UniqueName: \"kubernetes.io/projected/fdef514d-2dff-445a-8952-9878f932f251-kube-api-access-dpnvh\") pod \"fdef514d-2dff-445a-8952-9878f932f251\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " Jul 18 02:00:38.997303 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.997052 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-console-config\") pod \"fdef514d-2dff-445a-8952-9878f932f251\" (UID: \"fdef514d-2dff-445a-8952-9878f932f251\") " Jul 18 02:00:38.997484 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.997440 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-service-ca" (OuterVolumeSpecName: "service-ca") pod "fdef514d-2dff-445a-8952-9878f932f251" (UID: "fdef514d-2dff-445a-8952-9878f932f251"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:00:38.997484 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.997468 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fdef514d-2dff-445a-8952-9878f932f251" (UID: "fdef514d-2dff-445a-8952-9878f932f251"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:00:38.997484 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.997470 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fdef514d-2dff-445a-8952-9878f932f251" (UID: "fdef514d-2dff-445a-8952-9878f932f251"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:00:38.997680 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.997577 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-console-config" (OuterVolumeSpecName: "console-config") pod "fdef514d-2dff-445a-8952-9878f932f251" (UID: "fdef514d-2dff-445a-8952-9878f932f251"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:00:38.999008 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.998987 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fdef514d-2dff-445a-8952-9878f932f251" (UID: "fdef514d-2dff-445a-8952-9878f932f251"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:00:38.999559 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.999534 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fdef514d-2dff-445a-8952-9878f932f251" (UID: "fdef514d-2dff-445a-8952-9878f932f251"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:00:38.999559 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:38.999543 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdef514d-2dff-445a-8952-9878f932f251-kube-api-access-dpnvh" (OuterVolumeSpecName: "kube-api-access-dpnvh") pod "fdef514d-2dff-445a-8952-9878f932f251" (UID: "fdef514d-2dff-445a-8952-9878f932f251"). InnerVolumeSpecName "kube-api-access-dpnvh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:00:39.098544 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.098508 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-serving-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:00:39.098544 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.098539 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-oauth-serving-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:00:39.098544 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.098550 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-service-ca\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:00:39.098774 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.098558 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-trusted-ca-bundle\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:00:39.098774 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.098568 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fdef514d-2dff-445a-8952-9878f932f251-console-oauth-config\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:00:39.098774 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.098576 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpnvh\" (UniqueName: \"kubernetes.io/projected/fdef514d-2dff-445a-8952-9878f932f251-kube-api-access-dpnvh\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:00:39.098774 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.098586 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fdef514d-2dff-445a-8952-9878f932f251-console-config\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:00:39.659008 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.658950 2573 generic.go:356] "Generic (PLEG): container finished" podID="fdef514d-2dff-445a-8952-9878f932f251" containerID="34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3" exitCode=0 Jul 18 02:00:39.659008 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.659017 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b75964cff-njh2t" Jul 18 02:00:39.659492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.659043 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b75964cff-njh2t" event={"ID":"fdef514d-2dff-445a-8952-9878f932f251","Type":"ContainerDied","Data":"34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3"} Jul 18 02:00:39.659492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.659098 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b75964cff-njh2t" event={"ID":"fdef514d-2dff-445a-8952-9878f932f251","Type":"ContainerDied","Data":"5d1e409be16ef71ec0d956e1d26b7a32c628467fdfa08add2f3965743e81c6fd"} Jul 18 02:00:39.659492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.659119 2573 scope.go:117] "RemoveContainer" containerID="34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3" Jul 18 02:00:39.667427 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.667403 2573 scope.go:117] "RemoveContainer" containerID="34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3" Jul 18 02:00:39.667671 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:00:39.667652 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3\": container with ID starting with 34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3 not found: ID does not exist" containerID="34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3" Jul 18 02:00:39.667736 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.667683 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3"} err="failed to get container status \"34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3\": rpc error: code = NotFound desc = could not find container \"34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3\": container with ID starting with 34756eefc7dd6f4c58a360dbeec9bdf19c87eaa194ca88564bf7d66bc1b511c3 not found: ID does not exist" Jul 18 02:00:39.680142 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.680113 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b75964cff-njh2t"] Jul 18 02:00:39.682045 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:39.682022 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b75964cff-njh2t"] Jul 18 02:00:40.720790 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:40.720754 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdef514d-2dff-445a-8952-9878f932f251" path="/var/lib/kubelet/pods/fdef514d-2dff-445a-8952-9878f932f251/volumes" Jul 18 02:00:46.442404 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:46.442360 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:46.555528 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:46.555489 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:00:47.081050 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.081020 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb"] Jul 18 02:00:47.081275 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.081262 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdef514d-2dff-445a-8952-9878f932f251" containerName="console" Jul 18 02:00:47.081319 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.081277 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdef514d-2dff-445a-8952-9878f932f251" containerName="console" Jul 18 02:00:47.081319 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.081317 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdef514d-2dff-445a-8952-9878f932f251" containerName="console" Jul 18 02:00:47.083345 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.083330 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.085715 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.085693 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 02:00:47.085858 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.085787 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 02:00:47.086534 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.086518 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-29xcj\"" Jul 18 02:00:47.093756 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.093736 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb"] Jul 18 02:00:47.152615 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.152576 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh797\" (UniqueName: \"kubernetes.io/projected/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-kube-api-access-fh797\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.152615 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.152626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.152831 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.152656 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.253226 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.253185 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh797\" (UniqueName: \"kubernetes.io/projected/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-kube-api-access-fh797\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.253414 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.253250 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.253414 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.253285 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.253660 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.253641 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.253700 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.253664 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.262089 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.262055 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh797\" (UniqueName: \"kubernetes.io/projected/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-kube-api-access-fh797\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.391922 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.391888 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:00:47.519309 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.519278 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb"] Jul 18 02:00:47.522246 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:00:47.522220 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70350ae6_47bb_42ad_b6bc_bf1bd8f37a5a.slice/crio-74888d743fefad9ff5b5a70bca61cf979ea88f4a8d268a3ca0431b80efdbf059 WatchSource:0}: Error finding container 74888d743fefad9ff5b5a70bca61cf979ea88f4a8d268a3ca0431b80efdbf059: Status 404 returned error can't find the container with id 74888d743fefad9ff5b5a70bca61cf979ea88f4a8d268a3ca0431b80efdbf059 Jul 18 02:00:47.690526 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:47.690439 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" event={"ID":"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a","Type":"ContainerStarted","Data":"74888d743fefad9ff5b5a70bca61cf979ea88f4a8d268a3ca0431b80efdbf059"} Jul 18 02:00:54.718309 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:54.718269 2573 generic.go:356] "Generic (PLEG): container finished" podID="70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" containerID="5ec7c768179e082bec000513dd92c6d2bed1e765f1f69888362ef21b703c5ffa" exitCode=0 Jul 18 02:00:54.721297 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:54.721264 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" event={"ID":"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a","Type":"ContainerDied","Data":"5ec7c768179e082bec000513dd92c6d2bed1e765f1f69888362ef21b703c5ffa"} Jul 18 02:00:56.725521 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:56.725437 2573 generic.go:356] "Generic (PLEG): container finished" podID="70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" containerID="e102c2123b6f91992c6186594e9f9b3f210d42835660f5b7ceaf0db9347becea" exitCode=0 Jul 18 02:00:56.725521 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:00:56.725496 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" event={"ID":"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a","Type":"ContainerDied","Data":"e102c2123b6f91992c6186594e9f9b3f210d42835660f5b7ceaf0db9347becea"} Jul 18 02:01:03.756881 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:03.756848 2573 generic.go:356] "Generic (PLEG): container finished" podID="70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" containerID="c0529803cf73f7fc1fa1203a61cc55faae826aff773344cf2bd6066c3e42149e" exitCode=0 Jul 18 02:01:03.757272 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:03.756940 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" event={"ID":"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a","Type":"ContainerDied","Data":"c0529803cf73f7fc1fa1203a61cc55faae826aff773344cf2bd6066c3e42149e"} Jul 18 02:01:04.880359 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:04.880336 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:01:04.994104 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:04.994072 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh797\" (UniqueName: \"kubernetes.io/projected/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-kube-api-access-fh797\") pod \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " Jul 18 02:01:04.994258 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:04.994141 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-util\") pod \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " Jul 18 02:01:04.994258 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:04.994168 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-bundle\") pod \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\" (UID: \"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a\") " Jul 18 02:01:04.994928 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:04.994895 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-bundle" (OuterVolumeSpecName: "bundle") pod "70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" (UID: "70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:01:04.996333 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:04.996310 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-kube-api-access-fh797" (OuterVolumeSpecName: "kube-api-access-fh797") pod "70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" (UID: "70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a"). InnerVolumeSpecName "kube-api-access-fh797". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:01:04.999352 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:04.999330 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-util" (OuterVolumeSpecName: "util") pod "70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" (UID: "70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:01:05.095607 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.095522 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-bundle\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:01:05.095607 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.095553 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh797\" (UniqueName: \"kubernetes.io/projected/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-kube-api-access-fh797\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:01:05.095607 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.095566 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a-util\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:01:05.286861 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.286824 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:05.362191 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.362115 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv"] Jul 18 02:01:05.362351 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.362340 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" containerName="util" Jul 18 02:01:05.362394 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.362353 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" containerName="util" Jul 18 02:01:05.362394 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.362368 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" containerName="pull" Jul 18 02:01:05.362394 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.362374 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" containerName="pull" Jul 18 02:01:05.362394 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.362384 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" containerName="extract" Jul 18 02:01:05.362394 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.362389 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" containerName="extract" Jul 18 02:01:05.362554 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.362426 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a" containerName="extract" Jul 18 02:01:05.366137 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.366118 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.368519 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.368492 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 18 02:01:05.369223 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.369207 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 18 02:01:05.369321 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.369240 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 18 02:01:05.369321 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.369249 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 18 02:01:05.378135 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.378111 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv"] Jul 18 02:01:05.388038 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.388010 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:05.397487 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.397464 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8"] Jul 18 02:01:05.401307 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.401287 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.403501 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.403483 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 18 02:01:05.403591 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.403484 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 18 02:01:05.403591 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.403565 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 18 02:01:05.404031 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.404015 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 18 02:01:05.413616 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.413587 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8"] Jul 18 02:01:05.457856 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.457821 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:05.498036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.497997 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xckq9\" (UniqueName: \"kubernetes.io/projected/4f2faea6-11d1-48b9-a840-f80ebcdf313f-kube-api-access-xckq9\") pod \"klusterlet-addon-workmgr-7b5c5845bc-qbgvv\" (UID: \"4f2faea6-11d1-48b9-a840-f80ebcdf313f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.498158 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.498056 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8vss\" (UniqueName: \"kubernetes.io/projected/4e35917b-0bf3-48be-b4cb-1a47007f6542-kube-api-access-d8vss\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.498158 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.498091 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-ca\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.498158 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.498108 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.498158 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.498141 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f2faea6-11d1-48b9-a840-f80ebcdf313f-tmp\") pod \"klusterlet-addon-workmgr-7b5c5845bc-qbgvv\" (UID: \"4f2faea6-11d1-48b9-a840-f80ebcdf313f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.498299 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.498232 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4f2faea6-11d1-48b9-a840-f80ebcdf313f-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b5c5845bc-qbgvv\" (UID: \"4f2faea6-11d1-48b9-a840-f80ebcdf313f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.498299 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.498265 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e35917b-0bf3-48be-b4cb-1a47007f6542-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.498369 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.498291 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-hub\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.498369 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.498321 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.599432 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.599395 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xckq9\" (UniqueName: \"kubernetes.io/projected/4f2faea6-11d1-48b9-a840-f80ebcdf313f-kube-api-access-xckq9\") pod \"klusterlet-addon-workmgr-7b5c5845bc-qbgvv\" (UID: \"4f2faea6-11d1-48b9-a840-f80ebcdf313f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.599432 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.599433 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8vss\" (UniqueName: \"kubernetes.io/projected/4e35917b-0bf3-48be-b4cb-1a47007f6542-kube-api-access-d8vss\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.599650 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.599463 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-ca\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.599650 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.599576 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.606164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.599992 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f2faea6-11d1-48b9-a840-f80ebcdf313f-tmp\") pod \"klusterlet-addon-workmgr-7b5c5845bc-qbgvv\" (UID: \"4f2faea6-11d1-48b9-a840-f80ebcdf313f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.606164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.600071 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4f2faea6-11d1-48b9-a840-f80ebcdf313f-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b5c5845bc-qbgvv\" (UID: \"4f2faea6-11d1-48b9-a840-f80ebcdf313f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.606164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.600116 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e35917b-0bf3-48be-b4cb-1a47007f6542-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.606164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.600157 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-hub\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.606164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.600448 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f2faea6-11d1-48b9-a840-f80ebcdf313f-tmp\") pod \"klusterlet-addon-workmgr-7b5c5845bc-qbgvv\" (UID: \"4f2faea6-11d1-48b9-a840-f80ebcdf313f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.606164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.600511 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.606164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.601105 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e35917b-0bf3-48be-b4cb-1a47007f6542-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.606164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.603466 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.607757 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.607704 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-ca\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.607966 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.607939 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.607966 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.608021 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4f2faea6-11d1-48b9-a840-f80ebcdf313f-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b5c5845bc-qbgvv\" (UID: \"4f2faea6-11d1-48b9-a840-f80ebcdf313f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.608478 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.608443 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4e35917b-0bf3-48be-b4cb-1a47007f6542-hub\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.608562 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.608480 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8vss\" (UniqueName: \"kubernetes.io/projected/4e35917b-0bf3-48be-b4cb-1a47007f6542-kube-api-access-d8vss\") pod \"cluster-proxy-proxy-agent-6d695b56f5-ptms8\" (UID: \"4e35917b-0bf3-48be-b4cb-1a47007f6542\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.609263 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.609238 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xckq9\" (UniqueName: \"kubernetes.io/projected/4f2faea6-11d1-48b9-a840-f80ebcdf313f-kube-api-access-xckq9\") pod \"klusterlet-addon-workmgr-7b5c5845bc-qbgvv\" (UID: \"4f2faea6-11d1-48b9-a840-f80ebcdf313f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.675623 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.675577 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:05.720375 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.720332 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" Jul 18 02:01:05.768703 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.768664 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" event={"ID":"70350ae6-47bb-42ad-b6bc-bf1bd8f37a5a","Type":"ContainerDied","Data":"74888d743fefad9ff5b5a70bca61cf979ea88f4a8d268a3ca0431b80efdbf059"} Jul 18 02:01:05.768886 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.768713 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540llckb" Jul 18 02:01:05.768886 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.768718 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74888d743fefad9ff5b5a70bca61cf979ea88f4a8d268a3ca0431b80efdbf059" Jul 18 02:01:05.812843 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.812814 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv"] Jul 18 02:01:05.815212 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:01:05.815170 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2faea6_11d1_48b9_a840_f80ebcdf313f.slice/crio-f44637ec087ed1280392345a2ff28e562b6063f3a4de6a0f4fbae62a6893d250 WatchSource:0}: Error finding container f44637ec087ed1280392345a2ff28e562b6063f3a4de6a0f4fbae62a6893d250: Status 404 returned error can't find the container with id f44637ec087ed1280392345a2ff28e562b6063f3a4de6a0f4fbae62a6893d250 Jul 18 02:01:05.856229 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:05.856199 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8"] Jul 18 02:01:05.859310 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:01:05.859284 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e35917b_0bf3_48be_b4cb_1a47007f6542.slice/crio-f059a4ecda176fa30e72b5d39c7874fb7dbf6381711ba56685f1be762b67fb8d WatchSource:0}: Error finding container f059a4ecda176fa30e72b5d39c7874fb7dbf6381711ba56685f1be762b67fb8d: Status 404 returned error can't find the container with id f059a4ecda176fa30e72b5d39c7874fb7dbf6381711ba56685f1be762b67fb8d Jul 18 02:01:06.777911 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:06.777815 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" event={"ID":"4e35917b-0bf3-48be-b4cb-1a47007f6542","Type":"ContainerStarted","Data":"f059a4ecda176fa30e72b5d39c7874fb7dbf6381711ba56685f1be762b67fb8d"} Jul 18 02:01:06.780375 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:06.780288 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" event={"ID":"4f2faea6-11d1-48b9-a840-f80ebcdf313f","Type":"ContainerStarted","Data":"f44637ec087ed1280392345a2ff28e562b6063f3a4de6a0f4fbae62a6893d250"} Jul 18 02:01:08.052185 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:08.052114 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:08.892096 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:08.892042 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:11.662201 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:11.662161 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:11.763163 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:11.763129 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:11.803413 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:11.803367 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" event={"ID":"4e35917b-0bf3-48be-b4cb-1a47007f6542","Type":"ContainerStarted","Data":"4c40515dda72c4cda4d158cb07f8473bd71db2c190a15bb154e8ee0f7d02d788"} Jul 18 02:01:11.803545 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:11.803425 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" event={"ID":"4e35917b-0bf3-48be-b4cb-1a47007f6542","Type":"ContainerStarted","Data":"fa0bd7c537b8b6aca9729cb604061c2c9e7748f3d6ec46f5c305072d665c9042"} Jul 18 02:01:11.804945 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:11.804909 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" event={"ID":"4f2faea6-11d1-48b9-a840-f80ebcdf313f","Type":"ContainerStarted","Data":"354d6071d7737ce637b9414e2202976ea3ae8ea77f13b77b5ea3a2bee6d0f972"} Jul 18 02:01:11.805142 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:11.805126 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:11.807030 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:11.807010 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" Jul 18 02:01:11.822615 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:11.822551 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5c5845bc-qbgvv" podStartSLOduration=1.116211785 podStartE2EDuration="6.822533827s" totalImagesPullingTime="5.706322042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:11.822043698 +0000 UTC m=+259.610380281" watchObservedRunningTime="2026-07-18 02:01:11.822533827 +0000 UTC m=+259.610870409" Jul 18 02:01:12.810344 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:12.810299 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" event={"ID":"4e35917b-0bf3-48be-b4cb-1a47007f6542","Type":"ContainerStarted","Data":"ff952029346dae8c95b769407d3b0c7ec5a4fbbc197346bd7c0a275936567bf0"} Jul 18 02:01:12.829111 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:12.829062 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d695b56f5-ptms8" podStartSLOduration=2.15220083 podStartE2EDuration="7.829047237s" totalImagesPullingTime="5.676846407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:12.828223718 +0000 UTC m=+260.616560302" watchObservedRunningTime="2026-07-18 02:01:12.829047237 +0000 UTC m=+260.617383820" Jul 18 02:01:14.055151 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.055118 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:14.113895 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.113851 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:14.628600 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.628559 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:14.735035 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.734999 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:14.762394 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.762359 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl"] Jul 18 02:01:14.772673 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.772649 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:14.775207 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.775184 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 02:01:14.775308 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.775235 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 02:01:14.775999 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.775983 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-29xcj\"" Jul 18 02:01:14.780064 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.780039 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl"] Jul 18 02:01:14.879250 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.879161 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:14.879250 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.879205 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm7fr\" (UniqueName: \"kubernetes.io/projected/dd647623-1cce-467c-90a7-cc36b5de573b-kube-api-access-gm7fr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:14.879437 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.879286 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:14.980136 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.980095 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:14.980311 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.980160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:14.980311 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.980190 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm7fr\" (UniqueName: \"kubernetes.io/projected/dd647623-1cce-467c-90a7-cc36b5de573b-kube-api-access-gm7fr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:14.980476 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.980453 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:14.980535 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.980477 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:14.998563 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:14.998532 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm7fr\" (UniqueName: \"kubernetes.io/projected/dd647623-1cce-467c-90a7-cc36b5de573b-kube-api-access-gm7fr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:15.082638 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:15.082598 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:15.223328 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:15.223300 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl"] Jul 18 02:01:15.225703 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:01:15.225677 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd647623_1cce_467c_90a7_cc36b5de573b.slice/crio-2de2166aa510f0592a4b8163417e0e414a0d1b4270440c23be7a4dd2c0cc1657 WatchSource:0}: Error finding container 2de2166aa510f0592a4b8163417e0e414a0d1b4270440c23be7a4dd2c0cc1657: Status 404 returned error can't find the container with id 2de2166aa510f0592a4b8163417e0e414a0d1b4270440c23be7a4dd2c0cc1657 Jul 18 02:01:15.823050 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:15.823016 2573 generic.go:356] "Generic (PLEG): container finished" podID="dd647623-1cce-467c-90a7-cc36b5de573b" containerID="4a3d14047923ec664a3955cd0b23d1a46f1e86c36921424435d9320ae5b024d8" exitCode=0 Jul 18 02:01:15.823238 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:15.823075 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" event={"ID":"dd647623-1cce-467c-90a7-cc36b5de573b","Type":"ContainerDied","Data":"4a3d14047923ec664a3955cd0b23d1a46f1e86c36921424435d9320ae5b024d8"} Jul 18 02:01:15.823238 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:15.823113 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" event={"ID":"dd647623-1cce-467c-90a7-cc36b5de573b","Type":"ContainerStarted","Data":"2de2166aa510f0592a4b8163417e0e414a0d1b4270440c23be7a4dd2c0cc1657"} Jul 18 02:01:16.070036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:16.070001 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:16.453078 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:16.453042 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:18.479802 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:18.479714 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:18.582902 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:18.582872 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:18.838024 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:18.837921 2573 generic.go:356] "Generic (PLEG): container finished" podID="dd647623-1cce-467c-90a7-cc36b5de573b" containerID="f0cfa1559b2e8929d4bab57c90433a63ba076672c173a3f5c5bf5534f0386eff" exitCode=0 Jul 18 02:01:18.838163 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:18.838014 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" event={"ID":"dd647623-1cce-467c-90a7-cc36b5de573b","Type":"ContainerDied","Data":"f0cfa1559b2e8929d4bab57c90433a63ba076672c173a3f5c5bf5534f0386eff"} Jul 18 02:01:19.844345 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:19.844308 2573 generic.go:356] "Generic (PLEG): container finished" podID="dd647623-1cce-467c-90a7-cc36b5de573b" containerID="f273dcebaab92e6bc148f060a2177e1d703c044b76ba6027506680078f8921b5" exitCode=0 Jul 18 02:01:19.844727 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:19.844378 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" event={"ID":"dd647623-1cce-467c-90a7-cc36b5de573b","Type":"ContainerDied","Data":"f273dcebaab92e6bc148f060a2177e1d703c044b76ba6027506680078f8921b5"} Jul 18 02:01:20.973135 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:20.973113 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:21.124124 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.124093 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-bundle\") pod \"dd647623-1cce-467c-90a7-cc36b5de573b\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " Jul 18 02:01:21.124307 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.124146 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm7fr\" (UniqueName: \"kubernetes.io/projected/dd647623-1cce-467c-90a7-cc36b5de573b-kube-api-access-gm7fr\") pod \"dd647623-1cce-467c-90a7-cc36b5de573b\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " Jul 18 02:01:21.124307 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.124192 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-util\") pod \"dd647623-1cce-467c-90a7-cc36b5de573b\" (UID: \"dd647623-1cce-467c-90a7-cc36b5de573b\") " Jul 18 02:01:21.124565 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.124537 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-bundle" (OuterVolumeSpecName: "bundle") pod "dd647623-1cce-467c-90a7-cc36b5de573b" (UID: "dd647623-1cce-467c-90a7-cc36b5de573b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:01:21.126239 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.126215 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd647623-1cce-467c-90a7-cc36b5de573b-kube-api-access-gm7fr" (OuterVolumeSpecName: "kube-api-access-gm7fr") pod "dd647623-1cce-467c-90a7-cc36b5de573b" (UID: "dd647623-1cce-467c-90a7-cc36b5de573b"). InnerVolumeSpecName "kube-api-access-gm7fr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:01:21.128520 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.128487 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-util" (OuterVolumeSpecName: "util") pod "dd647623-1cce-467c-90a7-cc36b5de573b" (UID: "dd647623-1cce-467c-90a7-cc36b5de573b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:01:21.225308 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.225253 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-bundle\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:01:21.225308 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.225306 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm7fr\" (UniqueName: \"kubernetes.io/projected/dd647623-1cce-467c-90a7-cc36b5de573b-kube-api-access-gm7fr\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:01:21.225308 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.225315 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd647623-1cce-467c-90a7-cc36b5de573b-util\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:01:21.418101 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.418003 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:21.518934 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.518897 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:21.852728 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.852633 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" event={"ID":"dd647623-1cce-467c-90a7-cc36b5de573b","Type":"ContainerDied","Data":"2de2166aa510f0592a4b8163417e0e414a0d1b4270440c23be7a4dd2c0cc1657"} Jul 18 02:01:21.852728 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.852664 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlzbl" Jul 18 02:01:21.852728 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:21.852709 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2de2166aa510f0592a4b8163417e0e414a0d1b4270440c23be7a4dd2c0cc1657" Jul 18 02:01:24.870865 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:24.870833 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:25.962041 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:25.962010 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:26.056314 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:26.056285 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:31.959046 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:31.959011 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:32.183691 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:32.183657 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:33.380421 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.380386 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j"] Jul 18 02:01:33.380897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.380749 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd647623-1cce-467c-90a7-cc36b5de573b" containerName="util" Jul 18 02:01:33.380897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.380766 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd647623-1cce-467c-90a7-cc36b5de573b" containerName="util" Jul 18 02:01:33.380897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.380777 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd647623-1cce-467c-90a7-cc36b5de573b" containerName="pull" Jul 18 02:01:33.380897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.380785 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd647623-1cce-467c-90a7-cc36b5de573b" containerName="pull" Jul 18 02:01:33.380897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.380796 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd647623-1cce-467c-90a7-cc36b5de573b" containerName="extract" Jul 18 02:01:33.380897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.380804 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd647623-1cce-467c-90a7-cc36b5de573b" containerName="extract" Jul 18 02:01:33.380897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.380867 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd647623-1cce-467c-90a7-cc36b5de573b" containerName="extract" Jul 18 02:01:33.383152 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.383132 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.385482 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.385464 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 02:01:33.386268 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.386236 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 02:01:33.386343 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.386280 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-29xcj\"" Jul 18 02:01:33.391677 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.391653 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j"] Jul 18 02:01:33.426941 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.426903 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.427133 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.426948 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nfl8\" (UniqueName: \"kubernetes.io/projected/2e357d73-9051-423a-ab6a-5f4e4a59da1f-kube-api-access-8nfl8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.427133 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.427046 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.528185 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.528152 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.528371 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.528196 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nfl8\" (UniqueName: \"kubernetes.io/projected/2e357d73-9051-423a-ab6a-5f4e4a59da1f-kube-api-access-8nfl8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.528371 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.528245 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.528600 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.528576 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.528600 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.528591 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.536213 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.536185 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nfl8\" (UniqueName: \"kubernetes.io/projected/2e357d73-9051-423a-ab6a-5f4e4a59da1f-kube-api-access-8nfl8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.682550 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.682466 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:33.692279 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.692255 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:33.821718 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.821620 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j"] Jul 18 02:01:33.824264 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:01:33.824236 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e357d73_9051_423a_ab6a_5f4e4a59da1f.slice/crio-8e54ecd2d387e3f97c2f152ac883c673e8b546b603093392bdc91806753f2e2b WatchSource:0}: Error finding container 8e54ecd2d387e3f97c2f152ac883c673e8b546b603093392bdc91806753f2e2b: Status 404 returned error can't find the container with id 8e54ecd2d387e3f97c2f152ac883c673e8b546b603093392bdc91806753f2e2b Jul 18 02:01:33.900435 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.900401 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" event={"ID":"2e357d73-9051-423a-ab6a-5f4e4a59da1f","Type":"ContainerStarted","Data":"a082a7111e70f8777337fa6cdd789195d63ce45674f2eb8a0cfe72454cfbbd13"} Jul 18 02:01:33.900603 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:33.900452 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" event={"ID":"2e357d73-9051-423a-ab6a-5f4e4a59da1f","Type":"ContainerStarted","Data":"8e54ecd2d387e3f97c2f152ac883c673e8b546b603093392bdc91806753f2e2b"} Jul 18 02:01:34.905009 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:34.904955 2573 generic.go:356] "Generic (PLEG): container finished" podID="2e357d73-9051-423a-ab6a-5f4e4a59da1f" containerID="a082a7111e70f8777337fa6cdd789195d63ce45674f2eb8a0cfe72454cfbbd13" exitCode=0 Jul 18 02:01:34.905393 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:34.905044 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" event={"ID":"2e357d73-9051-423a-ab6a-5f4e4a59da1f","Type":"ContainerDied","Data":"a082a7111e70f8777337fa6cdd789195d63ce45674f2eb8a0cfe72454cfbbd13"} Jul 18 02:01:35.910333 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:35.910298 2573 generic.go:356] "Generic (PLEG): container finished" podID="2e357d73-9051-423a-ab6a-5f4e4a59da1f" containerID="d9606d7ec474801399981c27bdecc06175d4564780a7ca08b7efbb38e077f507" exitCode=0 Jul 18 02:01:35.910698 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:35.910379 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" event={"ID":"2e357d73-9051-423a-ab6a-5f4e4a59da1f","Type":"ContainerDied","Data":"d9606d7ec474801399981c27bdecc06175d4564780a7ca08b7efbb38e077f507"} Jul 18 02:01:36.915521 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:36.915486 2573 generic.go:356] "Generic (PLEG): container finished" podID="2e357d73-9051-423a-ab6a-5f4e4a59da1f" containerID="daee47876873dad8950e198756e10268dd0cbc3a243ac98eef191a5e7ac4cdc0" exitCode=0 Jul 18 02:01:36.915904 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:36.915558 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" event={"ID":"2e357d73-9051-423a-ab6a-5f4e4a59da1f","Type":"ContainerDied","Data":"daee47876873dad8950e198756e10268dd0cbc3a243ac98eef191a5e7ac4cdc0"} Jul 18 02:01:38.055929 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.055902 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:38.162882 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.162849 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nfl8\" (UniqueName: \"kubernetes.io/projected/2e357d73-9051-423a-ab6a-5f4e4a59da1f-kube-api-access-8nfl8\") pod \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " Jul 18 02:01:38.163096 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.162914 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-util\") pod \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " Jul 18 02:01:38.163096 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.162954 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-bundle\") pod \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\" (UID: \"2e357d73-9051-423a-ab6a-5f4e4a59da1f\") " Jul 18 02:01:38.163691 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.163664 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-bundle" (OuterVolumeSpecName: "bundle") pod "2e357d73-9051-423a-ab6a-5f4e4a59da1f" (UID: "2e357d73-9051-423a-ab6a-5f4e4a59da1f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:01:38.165060 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.165031 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e357d73-9051-423a-ab6a-5f4e4a59da1f-kube-api-access-8nfl8" (OuterVolumeSpecName: "kube-api-access-8nfl8") pod "2e357d73-9051-423a-ab6a-5f4e4a59da1f" (UID: "2e357d73-9051-423a-ab6a-5f4e4a59da1f"). InnerVolumeSpecName "kube-api-access-8nfl8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:01:38.169675 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.169622 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-util" (OuterVolumeSpecName: "util") pod "2e357d73-9051-423a-ab6a-5f4e4a59da1f" (UID: "2e357d73-9051-423a-ab6a-5f4e4a59da1f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:01:38.263589 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.263546 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-util\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:01:38.263589 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.263586 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e357d73-9051-423a-ab6a-5f4e4a59da1f-bundle\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:01:38.263772 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.263599 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nfl8\" (UniqueName: \"kubernetes.io/projected/2e357d73-9051-423a-ab6a-5f4e4a59da1f-kube-api-access-8nfl8\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:01:38.924824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.924799 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" Jul 18 02:01:38.924998 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.924799 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5p2j" event={"ID":"2e357d73-9051-423a-ab6a-5f4e4a59da1f","Type":"ContainerDied","Data":"8e54ecd2d387e3f97c2f152ac883c673e8b546b603093392bdc91806753f2e2b"} Jul 18 02:01:38.924998 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:38.924915 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e54ecd2d387e3f97c2f152ac883c673e8b546b603093392bdc91806753f2e2b" Jul 18 02:01:45.904289 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.904255 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:45.965689 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.965657 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7"] Jul 18 02:01:45.965992 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.965962 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e357d73-9051-423a-ab6a-5f4e4a59da1f" containerName="util" Jul 18 02:01:45.966080 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.965994 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e357d73-9051-423a-ab6a-5f4e4a59da1f" containerName="util" Jul 18 02:01:45.966080 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.966012 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e357d73-9051-423a-ab6a-5f4e4a59da1f" containerName="extract" Jul 18 02:01:45.966080 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.966020 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e357d73-9051-423a-ab6a-5f4e4a59da1f" containerName="extract" Jul 18 02:01:45.966080 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.966045 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e357d73-9051-423a-ab6a-5f4e4a59da1f" containerName="pull" Jul 18 02:01:45.966080 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.966053 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e357d73-9051-423a-ab6a-5f4e4a59da1f" containerName="pull" Jul 18 02:01:45.966369 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.966119 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e357d73-9051-423a-ab6a-5f4e4a59da1f" containerName="extract" Jul 18 02:01:45.979289 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.979263 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:45.982983 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.982953 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 18 02:01:45.983275 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.983256 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 18 02:01:45.984145 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.984125 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 18 02:01:45.984261 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.984141 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 18 02:01:45.984261 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.984174 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wln6l\"" Jul 18 02:01:45.984261 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.984144 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 18 02:01:45.997385 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:45.997360 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7"] Jul 18 02:01:46.012694 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.012666 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:46.023872 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.023845 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnd68\" (UniqueName: \"kubernetes.io/projected/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-kube-api-access-jnd68\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.024020 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.023878 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-manager-config\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.024020 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.023896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-metrics-cert\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.024020 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.023918 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-cert\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.125375 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.125330 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnd68\" (UniqueName: \"kubernetes.io/projected/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-kube-api-access-jnd68\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.125375 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.125379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-manager-config\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.125583 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.125488 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-metrics-cert\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.125583 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.125538 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-cert\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.126169 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.126143 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-manager-config\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.128059 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.128036 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-metrics-cert\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.128138 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.128086 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-cert\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.133834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.133813 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnd68\" (UniqueName: \"kubernetes.io/projected/1973b2b2-f1b1-48b4-9651-a7f652db1dc2-kube-api-access-jnd68\") pod \"lws-controller-manager-5b64b4b7d4-5zdw7\" (UID: \"1973b2b2-f1b1-48b4-9651-a7f652db1dc2\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.288741 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.288650 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:46.438351 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.438327 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7"] Jul 18 02:01:46.440814 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:01:46.440784 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1973b2b2_f1b1_48b4_9651_a7f652db1dc2.slice/crio-d84c03d2d1532a161ffcc81055dd80511b8f8da3d1b285438d0cc19adf918d6e WatchSource:0}: Error finding container d84c03d2d1532a161ffcc81055dd80511b8f8da3d1b285438d0cc19adf918d6e: Status 404 returned error can't find the container with id d84c03d2d1532a161ffcc81055dd80511b8f8da3d1b285438d0cc19adf918d6e Jul 18 02:01:46.956007 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:46.955949 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" event={"ID":"1973b2b2-f1b1-48b4-9651-a7f652db1dc2","Type":"ContainerStarted","Data":"d84c03d2d1532a161ffcc81055dd80511b8f8da3d1b285438d0cc19adf918d6e"} Jul 18 02:01:49.507013 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:49.506961 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:50.716195 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.716160 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6"] Jul 18 02:01:50.718455 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.718433 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.720967 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.720941 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 18 02:01:50.721118 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.721062 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-r5mvq\"" Jul 18 02:01:50.721118 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.721097 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 18 02:01:50.721118 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.721107 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 18 02:01:50.721595 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.721571 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 18 02:01:50.739791 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.739762 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6"] Jul 18 02:01:50.767183 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.767150 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-tmp\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.767183 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.767185 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.767402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.767215 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.767402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.767332 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb999\" (UniqueName: \"kubernetes.io/projected/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-kube-api-access-tb999\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.767402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.767364 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-manifests\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.867751 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.867656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-tmp\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.867751 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.867701 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.867751 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.867733 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.868070 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.867777 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb999\" (UniqueName: \"kubernetes.io/projected/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-kube-api-access-tb999\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.868070 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.867799 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-manifests\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.868256 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.868237 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-manifests\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.870077 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.870052 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-tmp\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.870330 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.870310 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.870446 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.870424 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.876823 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.876797 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb999\" (UniqueName: \"kubernetes.io/projected/081cb735-ba0b-4e64-910a-bb7e7ad6d55a-kube-api-access-tb999\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-hlgh6\" (UID: \"081cb735-ba0b-4e64-910a-bb7e7ad6d55a\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:50.972512 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.972479 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" event={"ID":"1973b2b2-f1b1-48b4-9651-a7f652db1dc2","Type":"ContainerStarted","Data":"216229fb7669e2e601fdaed623a50390e55830ed2d12a25d70089b898cd8a14c"} Jul 18 02:01:50.972704 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.972592 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:01:50.989017 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:50.988950 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" podStartSLOduration=1.846951875 podStartE2EDuration="5.988934789s" totalImagesPullingTime="4.141982914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:50.988047981 +0000 UTC m=+298.776384564" watchObservedRunningTime="2026-07-18 02:01:50.988934789 +0000 UTC m=+298.777271371" Jul 18 02:01:51.028403 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:51.028367 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:51.160578 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:51.160549 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6"] Jul 18 02:01:51.163966 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:01:51.163938 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081cb735_ba0b_4e64_910a_bb7e7ad6d55a.slice/crio-d6f446c0d6a2e1249845a9acb5c55817f20b6508c36180e9700b95c512145721 WatchSource:0}: Error finding container d6f446c0d6a2e1249845a9acb5c55817f20b6508c36180e9700b95c512145721: Status 404 returned error can't find the container with id d6f446c0d6a2e1249845a9acb5c55817f20b6508c36180e9700b95c512145721 Jul 18 02:01:51.333394 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:51.333359 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:01:51.978778 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:51.978738 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" event={"ID":"081cb735-ba0b-4e64-910a-bb7e7ad6d55a","Type":"ContainerStarted","Data":"d6f446c0d6a2e1249845a9acb5c55817f20b6508c36180e9700b95c512145721"} Jul 18 02:01:52.598773 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:52.598690 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:01:52.602180 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:52.602158 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:01:52.602407 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:52.602395 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 18 02:01:54.992689 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:54.992654 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" event={"ID":"081cb735-ba0b-4e64-910a-bb7e7ad6d55a","Type":"ContainerStarted","Data":"310cf77b2976477059cbc1e9fb0778eec54a8364456bb0dd180fd574e31da060"} Jul 18 02:01:55.997712 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:55.997623 2573 generic.go:356] "Generic (PLEG): container finished" podID="081cb735-ba0b-4e64-910a-bb7e7ad6d55a" containerID="310cf77b2976477059cbc1e9fb0778eec54a8364456bb0dd180fd574e31da060" exitCode=0 Jul 18 02:01:55.997712 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:55.997694 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" event={"ID":"081cb735-ba0b-4e64-910a-bb7e7ad6d55a","Type":"ContainerDied","Data":"310cf77b2976477059cbc1e9fb0778eec54a8364456bb0dd180fd574e31da060"} Jul 18 02:01:55.998177 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:55.998162 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:01:57.003133 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:57.003090 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" event={"ID":"081cb735-ba0b-4e64-910a-bb7e7ad6d55a","Type":"ContainerStarted","Data":"f172a85591cd7e63f0cf893df4221b7d2d0e4b0a38adf947ffa9eba495f75bdf"} Jul 18 02:01:57.003514 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:57.003225 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:01:57.023929 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:01:57.023887 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" podStartSLOduration=2.085065911 podStartE2EDuration="7.023872291s" totalImagesPullingTime="3.841993599s" totalInitContainerRuntime="1.096812781s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:57.022388878 +0000 UTC m=+304.810725460" watchObservedRunningTime="2026-07-18 02:01:57.023872291 +0000 UTC m=+304.812208873" Jul 18 02:02:01.981079 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:01.981046 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-5zdw7" Jul 18 02:02:08.009577 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:08.009547 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-hlgh6" Jul 18 02:02:10.064888 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:10.064850 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:10.195315 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:10.195275 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:10.807187 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:10.807148 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:10.900079 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:10.900045 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:10.996735 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:10.996694 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:11.369590 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:11.369557 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:11.482230 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:11.482194 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:19.728860 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:19.728786 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:19.863550 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:19.863513 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:20.205129 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:20.205099 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:20.325199 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:20.325160 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:21.026750 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:21.026706 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:21.121032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:21.120997 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:21.193463 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:21.193430 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:21.569260 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:21.569217 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:22.727863 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:22.727827 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:22.815121 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:22.815092 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:23.806235 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:23.806196 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:28.051689 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.051652 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:28.116537 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.116498 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:28.621440 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.617085 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d"] Jul 18 02:02:28.627372 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.627342 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.630129 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.630101 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 18 02:02:28.630282 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.630244 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-42kbk\"" Jul 18 02:02:28.630358 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.630338 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 18 02:02:28.631296 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.631275 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 18 02:02:28.633883 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.633859 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d"] Jul 18 02:02:28.657345 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.657310 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:28.682602 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.682569 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.682795 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.682609 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.682795 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.682692 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6z6f\" (UniqueName: \"kubernetes.io/projected/9e70ad11-ef7b-49b0-aafd-224381739a45-kube-api-access-t6z6f\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.682795 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.682744 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e70ad11-ef7b-49b0-aafd-224381739a45-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.682989 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.682828 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.682989 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.682879 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.682989 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.682905 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.682989 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.682936 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.682989 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.682965 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.783441 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.783405 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6z6f\" (UniqueName: \"kubernetes.io/projected/9e70ad11-ef7b-49b0-aafd-224381739a45-kube-api-access-t6z6f\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.783441 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.783441 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e70ad11-ef7b-49b0-aafd-224381739a45-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.783695 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.783471 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.783695 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.783510 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.783695 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.783529 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.783695 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.783551 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.783695 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.783566 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.783695 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.783602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.783695 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.783626 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.784064 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.783958 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.784064 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.784007 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.784064 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.784057 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.784214 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.784123 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.784311 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.784275 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e70ad11-ef7b-49b0-aafd-224381739a45-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.786232 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.786211 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.786437 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.786420 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.794343 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.794310 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e70ad11-ef7b-49b0-aafd-224381739a45-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.794539 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.794466 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6z6f\" (UniqueName: \"kubernetes.io/projected/9e70ad11-ef7b-49b0-aafd-224381739a45-kube-api-access-t6z6f\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p724d\" (UID: \"9e70ad11-ef7b-49b0-aafd-224381739a45\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.892409 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.892372 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:28.939425 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.939380 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:28.988084 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:28.988048 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:29.085550 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:29.085522 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d"] Jul 18 02:02:29.088157 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:02:29.088130 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e70ad11_ef7b_49b0_aafd_224381739a45.slice/crio-4733c37f759c6863fb658bd2634012ace5d66dcb2062762a1b8a21d83b0b853e WatchSource:0}: Error finding container 4733c37f759c6863fb658bd2634012ace5d66dcb2062762a1b8a21d83b0b853e: Status 404 returned error can't find the container with id 4733c37f759c6863fb658bd2634012ace5d66dcb2062762a1b8a21d83b0b853e Jul 18 02:02:29.133013 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:29.132957 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" event={"ID":"9e70ad11-ef7b-49b0-aafd-224381739a45","Type":"ContainerStarted","Data":"4733c37f759c6863fb658bd2634012ace5d66dcb2062762a1b8a21d83b0b853e"} Jul 18 02:02:32.028709 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:32.028629 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:32.082859 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:32.082818 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:02:32.082984 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:32.082909 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:02:32.083064 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:32.082959 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:02:32.148213 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:32.148170 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" event={"ID":"9e70ad11-ef7b-49b0-aafd-224381739a45","Type":"ContainerStarted","Data":"9f5168b6a5d4d883eaf3e9d2ec5186612d8c63b227db005eaf0c5110b37a3bb5"} Jul 18 02:02:32.167712 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:32.167647 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" podStartSLOduration=1.175236461 podStartE2EDuration="4.167627246s" totalImagesPullingTime="2.992390785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:32.166348914 +0000 UTC m=+339.954685498" watchObservedRunningTime="2026-07-18 02:02:32.167627246 +0000 UTC m=+339.955963830" Jul 18 02:02:32.406272 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:32.406239 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:32.939573 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:32.939522 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:33.946097 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:33.946052 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:33.980700 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:33.980659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:34.063121 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:34.063083 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:02:34.156550 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:34.156514 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:02:34.157687 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:02:34.157667 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p724d" Jul 18 02:03:11.432512 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.432419 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:11.529361 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.529314 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:11.572531 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.572494 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qcs8b"] Jul 18 02:03:11.578754 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.578725 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" Jul 18 02:03:11.581308 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.581281 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 02:03:11.581574 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.581548 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 02:03:11.581917 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.581895 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-7gws7\"" Jul 18 02:03:11.583996 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.583956 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qcs8b"] Jul 18 02:03:11.676112 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.676066 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:11.737721 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.737620 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njhmq\" (UniqueName: \"kubernetes.io/projected/24865d82-b77b-4a00-bc9c-59e2f5e4e8e1-kube-api-access-njhmq\") pod \"kuadrant-operator-catalog-qcs8b\" (UID: \"24865d82-b77b-4a00-bc9c-59e2f5e4e8e1\") " pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" Jul 18 02:03:11.839076 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.839032 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njhmq\" (UniqueName: \"kubernetes.io/projected/24865d82-b77b-4a00-bc9c-59e2f5e4e8e1-kube-api-access-njhmq\") pod \"kuadrant-operator-catalog-qcs8b\" (UID: \"24865d82-b77b-4a00-bc9c-59e2f5e4e8e1\") " pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" Jul 18 02:03:11.847334 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.847307 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njhmq\" (UniqueName: \"kubernetes.io/projected/24865d82-b77b-4a00-bc9c-59e2f5e4e8e1-kube-api-access-njhmq\") pod \"kuadrant-operator-catalog-qcs8b\" (UID: \"24865d82-b77b-4a00-bc9c-59e2f5e4e8e1\") " pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" Jul 18 02:03:11.890205 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.890164 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" Jul 18 02:03:11.936585 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:11.936550 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qcs8b"] Jul 18 02:03:12.024664 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:12.024620 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qcs8b"] Jul 18 02:03:12.026900 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:03:12.026867 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24865d82_b77b_4a00_bc9c_59e2f5e4e8e1.slice/crio-aa3d88ffc9c60b4dcbbfbd16aa48e00ba59cf2b7f9cf5f196dae351b6cd2b8ce WatchSource:0}: Error finding container aa3d88ffc9c60b4dcbbfbd16aa48e00ba59cf2b7f9cf5f196dae351b6cd2b8ce: Status 404 returned error can't find the container with id aa3d88ffc9c60b4dcbbfbd16aa48e00ba59cf2b7f9cf5f196dae351b6cd2b8ce Jul 18 02:03:12.148393 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:12.148355 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-26s7w"] Jul 18 02:03:12.151126 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:12.151107 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-26s7w" Jul 18 02:03:12.158637 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:12.158594 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-26s7w"] Jul 18 02:03:12.243168 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:12.243134 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgktl\" (UniqueName: \"kubernetes.io/projected/2b249822-89e4-46af-93d4-370a6935bc53-kube-api-access-pgktl\") pod \"kuadrant-operator-catalog-26s7w\" (UID: \"2b249822-89e4-46af-93d4-370a6935bc53\") " pod="kuadrant-system/kuadrant-operator-catalog-26s7w" Jul 18 02:03:12.315888 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:12.315798 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" event={"ID":"24865d82-b77b-4a00-bc9c-59e2f5e4e8e1","Type":"ContainerStarted","Data":"aa3d88ffc9c60b4dcbbfbd16aa48e00ba59cf2b7f9cf5f196dae351b6cd2b8ce"} Jul 18 02:03:12.344386 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:12.344343 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgktl\" (UniqueName: \"kubernetes.io/projected/2b249822-89e4-46af-93d4-370a6935bc53-kube-api-access-pgktl\") pod \"kuadrant-operator-catalog-26s7w\" (UID: \"2b249822-89e4-46af-93d4-370a6935bc53\") " pod="kuadrant-system/kuadrant-operator-catalog-26s7w" Jul 18 02:03:12.351779 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:12.351747 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgktl\" (UniqueName: \"kubernetes.io/projected/2b249822-89e4-46af-93d4-370a6935bc53-kube-api-access-pgktl\") pod \"kuadrant-operator-catalog-26s7w\" (UID: \"2b249822-89e4-46af-93d4-370a6935bc53\") " pod="kuadrant-system/kuadrant-operator-catalog-26s7w" Jul 18 02:03:12.462100 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:12.462045 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-26s7w" Jul 18 02:03:12.593817 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:12.593761 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-26s7w"] Jul 18 02:03:12.596836 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:03:12.596798 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b249822_89e4_46af_93d4_370a6935bc53.slice/crio-65d0da6a9635e38c373a990a17df87fa2b2256d881a6554b95fdbe6a8e2310e7 WatchSource:0}: Error finding container 65d0da6a9635e38c373a990a17df87fa2b2256d881a6554b95fdbe6a8e2310e7: Status 404 returned error can't find the container with id 65d0da6a9635e38c373a990a17df87fa2b2256d881a6554b95fdbe6a8e2310e7 Jul 18 02:03:13.323433 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:13.323394 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-26s7w" event={"ID":"2b249822-89e4-46af-93d4-370a6935bc53","Type":"ContainerStarted","Data":"65d0da6a9635e38c373a990a17df87fa2b2256d881a6554b95fdbe6a8e2310e7"} Jul 18 02:03:14.329638 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:14.329514 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" event={"ID":"24865d82-b77b-4a00-bc9c-59e2f5e4e8e1","Type":"ContainerStarted","Data":"7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716"} Jul 18 02:03:14.329638 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:14.329551 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" podUID="24865d82-b77b-4a00-bc9c-59e2f5e4e8e1" containerName="registry-server" containerID="cri-o://7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716" gracePeriod=2 Jul 18 02:03:14.331180 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:14.331152 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-26s7w" event={"ID":"2b249822-89e4-46af-93d4-370a6935bc53","Type":"ContainerStarted","Data":"34d6b38849bda422a249b528f0ca1efbaf3e3949373996f88a5b343adb75a6e5"} Jul 18 02:03:14.344287 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:14.344238 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" podStartSLOduration=1.321054918 podStartE2EDuration="3.344220217s" totalImagesPullingTime="2.023165299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:03:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:03:14.343484533 +0000 UTC m=+382.131821126" watchObservedRunningTime="2026-07-18 02:03:14.344220217 +0000 UTC m=+382.132556800" Jul 18 02:03:14.361492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:14.361434 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-26s7w" podStartSLOduration=0.907404741 podStartE2EDuration="2.361419209s" totalImagesPullingTime="1.454014468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:03:14.359102324 +0000 UTC m=+382.147438910" watchObservedRunningTime="2026-07-18 02:03:14.361419209 +0000 UTC m=+382.149755793" Jul 18 02:03:14.567153 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:14.567122 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" Jul 18 02:03:14.665695 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:14.665647 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njhmq\" (UniqueName: \"kubernetes.io/projected/24865d82-b77b-4a00-bc9c-59e2f5e4e8e1-kube-api-access-njhmq\") pod \"24865d82-b77b-4a00-bc9c-59e2f5e4e8e1\" (UID: \"24865d82-b77b-4a00-bc9c-59e2f5e4e8e1\") " Jul 18 02:03:14.667845 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:14.667817 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24865d82-b77b-4a00-bc9c-59e2f5e4e8e1-kube-api-access-njhmq" (OuterVolumeSpecName: "kube-api-access-njhmq") pod "24865d82-b77b-4a00-bc9c-59e2f5e4e8e1" (UID: "24865d82-b77b-4a00-bc9c-59e2f5e4e8e1"). InnerVolumeSpecName "kube-api-access-njhmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:03:14.767246 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:14.767207 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njhmq\" (UniqueName: \"kubernetes.io/projected/24865d82-b77b-4a00-bc9c-59e2f5e4e8e1-kube-api-access-njhmq\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:15.337145 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:15.337054 2573 generic.go:356] "Generic (PLEG): container finished" podID="24865d82-b77b-4a00-bc9c-59e2f5e4e8e1" containerID="7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716" exitCode=0 Jul 18 02:03:15.337145 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:15.337113 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" Jul 18 02:03:15.337606 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:15.337152 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" event={"ID":"24865d82-b77b-4a00-bc9c-59e2f5e4e8e1","Type":"ContainerDied","Data":"7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716"} Jul 18 02:03:15.337606 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:15.337233 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qcs8b" event={"ID":"24865d82-b77b-4a00-bc9c-59e2f5e4e8e1","Type":"ContainerDied","Data":"aa3d88ffc9c60b4dcbbfbd16aa48e00ba59cf2b7f9cf5f196dae351b6cd2b8ce"} Jul 18 02:03:15.337606 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:15.337255 2573 scope.go:117] "RemoveContainer" containerID="7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716" Jul 18 02:03:15.345784 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:15.345764 2573 scope.go:117] "RemoveContainer" containerID="7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716" Jul 18 02:03:15.346113 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:03:15.346096 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716\": container with ID starting with 7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716 not found: ID does not exist" containerID="7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716" Jul 18 02:03:15.346184 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:15.346124 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716"} err="failed to get container status \"7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716\": rpc error: code = NotFound desc = could not find container \"7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716\": container with ID starting with 7696af925fa8b1c278ab9050c7baea217a5295c28dda50b171eec2c182700716 not found: ID does not exist" Jul 18 02:03:15.354863 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:15.354831 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qcs8b"] Jul 18 02:03:15.360929 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:15.360902 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qcs8b"] Jul 18 02:03:16.722187 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:16.722150 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24865d82-b77b-4a00-bc9c-59e2f5e4e8e1" path="/var/lib/kubelet/pods/24865d82-b77b-4a00-bc9c-59e2f5e4e8e1/volumes" Jul 18 02:03:22.462779 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:22.462732 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-26s7w" Jul 18 02:03:22.463219 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:22.462813 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-26s7w" Jul 18 02:03:22.485564 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:22.485538 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-26s7w" Jul 18 02:03:23.393680 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:23.393647 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-26s7w" Jul 18 02:03:26.914824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:26.914786 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf"] Jul 18 02:03:26.915255 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:26.915082 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24865d82-b77b-4a00-bc9c-59e2f5e4e8e1" containerName="registry-server" Jul 18 02:03:26.915255 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:26.915094 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="24865d82-b77b-4a00-bc9c-59e2f5e4e8e1" containerName="registry-server" Jul 18 02:03:26.915255 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:26.915166 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="24865d82-b77b-4a00-bc9c-59e2f5e4e8e1" containerName="registry-server" Jul 18 02:03:26.917637 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:26.917620 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:26.919799 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:26.919779 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-p4c9m\"" Jul 18 02:03:26.926328 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:26.926300 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf"] Jul 18 02:03:27.071729 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.071686 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:27.071729 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.071731 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:27.071954 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.071824 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rqbc\" (UniqueName: \"kubernetes.io/projected/18429789-5590-4420-99cb-ada89afc4685-kube-api-access-8rqbc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:27.173105 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.173009 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:27.173105 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.173079 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rqbc\" (UniqueName: \"kubernetes.io/projected/18429789-5590-4420-99cb-ada89afc4685-kube-api-access-8rqbc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:27.173302 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.173131 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:27.173475 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.173459 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:27.173515 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.173459 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:27.181098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.181073 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rqbc\" (UniqueName: \"kubernetes.io/projected/18429789-5590-4420-99cb-ada89afc4685-kube-api-access-8rqbc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:27.227610 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.227560 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:27.357020 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.356980 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf"] Jul 18 02:03:27.360381 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:03:27.360354 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18429789_5590_4420_99cb_ada89afc4685.slice/crio-e14176b1c13335003b54cb89978b3f0b13e68d4f434c14d159b20c0ccf9714f6 WatchSource:0}: Error finding container e14176b1c13335003b54cb89978b3f0b13e68d4f434c14d159b20c0ccf9714f6: Status 404 returned error can't find the container with id e14176b1c13335003b54cb89978b3f0b13e68d4f434c14d159b20c0ccf9714f6 Jul 18 02:03:27.389846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.389809 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" event={"ID":"18429789-5590-4420-99cb-ada89afc4685","Type":"ContainerStarted","Data":"e14176b1c13335003b54cb89978b3f0b13e68d4f434c14d159b20c0ccf9714f6"} Jul 18 02:03:27.518513 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.518473 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm"] Jul 18 02:03:27.524751 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.524714 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.529243 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.529215 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm"] Jul 18 02:03:27.576418 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.576376 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.576602 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.576434 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.576602 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.576460 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwpjt\" (UniqueName: \"kubernetes.io/projected/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-kube-api-access-cwpjt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.677150 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.677111 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.677351 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.677166 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.677351 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.677194 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwpjt\" (UniqueName: \"kubernetes.io/projected/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-kube-api-access-cwpjt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.677495 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.677475 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.677605 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.677585 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.690232 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.690194 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwpjt\" (UniqueName: \"kubernetes.io/projected/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-kube-api-access-cwpjt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.835516 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.835417 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:27.965119 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:27.964953 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm"] Jul 18 02:03:27.967563 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:03:27.967532 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a77011_e9f3_453b_8fb1_1d3bc2c1ca22.slice/crio-f76f0c95d88e31876c9f6ddf4a8309bc073c5ce4c3dc1e2f7edabbff9c1a603e WatchSource:0}: Error finding container f76f0c95d88e31876c9f6ddf4a8309bc073c5ce4c3dc1e2f7edabbff9c1a603e: Status 404 returned error can't find the container with id f76f0c95d88e31876c9f6ddf4a8309bc073c5ce4c3dc1e2f7edabbff9c1a603e Jul 18 02:03:28.115500 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.115414 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc"] Jul 18 02:03:28.117898 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.117875 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.127575 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.127545 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc"] Jul 18 02:03:28.181176 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.181140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.181357 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.181291 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz2kt\" (UniqueName: \"kubernetes.io/projected/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-kube-api-access-zz2kt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.181357 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.181325 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.282502 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.282456 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz2kt\" (UniqueName: \"kubernetes.io/projected/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-kube-api-access-zz2kt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.282502 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.282504 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.282721 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.282534 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.282934 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.282914 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.283009 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.282930 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.290252 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.290219 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz2kt\" (UniqueName: \"kubernetes.io/projected/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-kube-api-access-zz2kt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.395366 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.395332 2573 generic.go:356] "Generic (PLEG): container finished" podID="18429789-5590-4420-99cb-ada89afc4685" containerID="86f47ae18caa59c77aa3227572c9c4c381c24206ea46bb877e6778515d5ddf8f" exitCode=0 Jul 18 02:03:28.395583 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.395419 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" event={"ID":"18429789-5590-4420-99cb-ada89afc4685","Type":"ContainerDied","Data":"86f47ae18caa59c77aa3227572c9c4c381c24206ea46bb877e6778515d5ddf8f"} Jul 18 02:03:28.396959 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.396925 2573 generic.go:356] "Generic (PLEG): container finished" podID="b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" containerID="0750ab2bde003eb499ee9f8ded4c5dce572bcd5a7c27c6464a5e74d6f1134e6f" exitCode=0 Jul 18 02:03:28.397089 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.397009 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" event={"ID":"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22","Type":"ContainerDied","Data":"0750ab2bde003eb499ee9f8ded4c5dce572bcd5a7c27c6464a5e74d6f1134e6f"} Jul 18 02:03:28.397089 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.397057 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" event={"ID":"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22","Type":"ContainerStarted","Data":"f76f0c95d88e31876c9f6ddf4a8309bc073c5ce4c3dc1e2f7edabbff9c1a603e"} Jul 18 02:03:28.432996 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.432937 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:28.519372 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.519339 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h"] Jul 18 02:03:28.523402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.523372 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.531139 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.531057 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h"] Jul 18 02:03:28.569083 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.569057 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc"] Jul 18 02:03:28.571240 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:03:28.571209 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccbd3263_6251_4e1c_8fe7_713e2fc39d53.slice/crio-6a18c35a14c28ad761cd501a2318e32dd2a7b791d6f8f786acd469c6144be4ae WatchSource:0}: Error finding container 6a18c35a14c28ad761cd501a2318e32dd2a7b791d6f8f786acd469c6144be4ae: Status 404 returned error can't find the container with id 6a18c35a14c28ad761cd501a2318e32dd2a7b791d6f8f786acd469c6144be4ae Jul 18 02:03:28.586662 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.585590 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.586662 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.585642 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88qzt\" (UniqueName: \"kubernetes.io/projected/de6109e1-4420-4de5-bff3-d3f957245170-kube-api-access-88qzt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.586662 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.585743 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.686884 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.686848 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.686884 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.686901 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.687156 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.686940 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88qzt\" (UniqueName: \"kubernetes.io/projected/de6109e1-4420-4de5-bff3-d3f957245170-kube-api-access-88qzt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.687316 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.687295 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.687380 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.687324 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.695917 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.695890 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88qzt\" (UniqueName: \"kubernetes.io/projected/de6109e1-4420-4de5-bff3-d3f957245170-kube-api-access-88qzt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.837014 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.836952 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:28.965156 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:28.965133 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h"] Jul 18 02:03:28.967489 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:03:28.967459 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde6109e1_4420_4de5_bff3_d3f957245170.slice/crio-b86bb7861a30b51e608df19c0d72d5f122b0dd765b28a34346409e559d481781 WatchSource:0}: Error finding container b86bb7861a30b51e608df19c0d72d5f122b0dd765b28a34346409e559d481781: Status 404 returned error can't find the container with id b86bb7861a30b51e608df19c0d72d5f122b0dd765b28a34346409e559d481781 Jul 18 02:03:29.403150 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:29.403117 2573 generic.go:356] "Generic (PLEG): container finished" podID="ccbd3263-6251-4e1c-8fe7-713e2fc39d53" containerID="ca36b319223305bf63ec6e4577486848194b9432a6e5008687b275c9b17392c2" exitCode=0 Jul 18 02:03:29.403323 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:29.403204 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" event={"ID":"ccbd3263-6251-4e1c-8fe7-713e2fc39d53","Type":"ContainerDied","Data":"ca36b319223305bf63ec6e4577486848194b9432a6e5008687b275c9b17392c2"} Jul 18 02:03:29.403323 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:29.403254 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" event={"ID":"ccbd3263-6251-4e1c-8fe7-713e2fc39d53","Type":"ContainerStarted","Data":"6a18c35a14c28ad761cd501a2318e32dd2a7b791d6f8f786acd469c6144be4ae"} Jul 18 02:03:29.404824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:29.404801 2573 generic.go:356] "Generic (PLEG): container finished" podID="de6109e1-4420-4de5-bff3-d3f957245170" containerID="c7f68c0a544189b715d5fe21d30b762ee0bb48deb28deb9ac5c385c197b84f12" exitCode=0 Jul 18 02:03:29.404911 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:29.404847 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" event={"ID":"de6109e1-4420-4de5-bff3-d3f957245170","Type":"ContainerDied","Data":"c7f68c0a544189b715d5fe21d30b762ee0bb48deb28deb9ac5c385c197b84f12"} Jul 18 02:03:29.404911 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:29.404903 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" event={"ID":"de6109e1-4420-4de5-bff3-d3f957245170","Type":"ContainerStarted","Data":"b86bb7861a30b51e608df19c0d72d5f122b0dd765b28a34346409e559d481781"} Jul 18 02:03:29.406965 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:29.406933 2573 generic.go:356] "Generic (PLEG): container finished" podID="18429789-5590-4420-99cb-ada89afc4685" containerID="96ad683f996714217887d99d66e622e831f728f2e86c34fe43e0ea26f50d51b6" exitCode=0 Jul 18 02:03:29.407071 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:29.406991 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" event={"ID":"18429789-5590-4420-99cb-ada89afc4685","Type":"ContainerDied","Data":"96ad683f996714217887d99d66e622e831f728f2e86c34fe43e0ea26f50d51b6"} Jul 18 02:03:30.414143 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:30.414109 2573 generic.go:356] "Generic (PLEG): container finished" podID="b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" containerID="0dee7d737c9bb527181a913ce2788ec64948a47699189aa790a8f994c5fc354c" exitCode=0 Jul 18 02:03:30.414584 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:30.414189 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" event={"ID":"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22","Type":"ContainerDied","Data":"0dee7d737c9bb527181a913ce2788ec64948a47699189aa790a8f994c5fc354c"} Jul 18 02:03:30.416490 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:30.416466 2573 generic.go:356] "Generic (PLEG): container finished" podID="18429789-5590-4420-99cb-ada89afc4685" containerID="15d9658bc58214c1ea9e0f4cedbed0aabf37b2addce8647f781dadea0f603c17" exitCode=0 Jul 18 02:03:30.416575 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:30.416518 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" event={"ID":"18429789-5590-4420-99cb-ada89afc4685","Type":"ContainerDied","Data":"15d9658bc58214c1ea9e0f4cedbed0aabf37b2addce8647f781dadea0f603c17"} Jul 18 02:03:31.423134 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.423099 2573 generic.go:356] "Generic (PLEG): container finished" podID="de6109e1-4420-4de5-bff3-d3f957245170" containerID="e436bcd7cb7c604286159534b6d567e6c2f70fde507a40b40d52a69cafa40fb5" exitCode=0 Jul 18 02:03:31.423538 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.423186 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" event={"ID":"de6109e1-4420-4de5-bff3-d3f957245170","Type":"ContainerDied","Data":"e436bcd7cb7c604286159534b6d567e6c2f70fde507a40b40d52a69cafa40fb5"} Jul 18 02:03:31.425522 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.425500 2573 generic.go:356] "Generic (PLEG): container finished" podID="b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" containerID="c16ca12f67f39d3f3b24276d1470c80a2493feb94da7e84bc0e75eb59fe29880" exitCode=0 Jul 18 02:03:31.425620 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.425557 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" event={"ID":"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22","Type":"ContainerDied","Data":"c16ca12f67f39d3f3b24276d1470c80a2493feb94da7e84bc0e75eb59fe29880"} Jul 18 02:03:31.655551 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.655520 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:31.715816 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.715722 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rqbc\" (UniqueName: \"kubernetes.io/projected/18429789-5590-4420-99cb-ada89afc4685-kube-api-access-8rqbc\") pod \"18429789-5590-4420-99cb-ada89afc4685\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " Jul 18 02:03:31.715816 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.715780 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-util\") pod \"18429789-5590-4420-99cb-ada89afc4685\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " Jul 18 02:03:31.715816 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.715801 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-bundle\") pod \"18429789-5590-4420-99cb-ada89afc4685\" (UID: \"18429789-5590-4420-99cb-ada89afc4685\") " Jul 18 02:03:31.716314 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.716289 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-bundle" (OuterVolumeSpecName: "bundle") pod "18429789-5590-4420-99cb-ada89afc4685" (UID: "18429789-5590-4420-99cb-ada89afc4685"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:03:31.717911 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.717883 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18429789-5590-4420-99cb-ada89afc4685-kube-api-access-8rqbc" (OuterVolumeSpecName: "kube-api-access-8rqbc") pod "18429789-5590-4420-99cb-ada89afc4685" (UID: "18429789-5590-4420-99cb-ada89afc4685"). InnerVolumeSpecName "kube-api-access-8rqbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:03:31.721370 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.721332 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-util" (OuterVolumeSpecName: "util") pod "18429789-5590-4420-99cb-ada89afc4685" (UID: "18429789-5590-4420-99cb-ada89afc4685"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:03:31.816873 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.816834 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rqbc\" (UniqueName: \"kubernetes.io/projected/18429789-5590-4420-99cb-ada89afc4685-kube-api-access-8rqbc\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:31.816873 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.816867 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-util\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:31.816873 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:31.816877 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18429789-5590-4420-99cb-ada89afc4685-bundle\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:32.432309 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.432271 2573 generic.go:356] "Generic (PLEG): container finished" podID="ccbd3263-6251-4e1c-8fe7-713e2fc39d53" containerID="c2926edd236a7b012a50b05ce5c2f0aac69262634c1101c99ad267ed05444311" exitCode=0 Jul 18 02:03:32.432716 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.432363 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" event={"ID":"ccbd3263-6251-4e1c-8fe7-713e2fc39d53","Type":"ContainerDied","Data":"c2926edd236a7b012a50b05ce5c2f0aac69262634c1101c99ad267ed05444311"} Jul 18 02:03:32.434584 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.434558 2573 generic.go:356] "Generic (PLEG): container finished" podID="de6109e1-4420-4de5-bff3-d3f957245170" containerID="ca6616a6d16cec33307abbec1b9438a3655c52fbc56d3857ec1510fd95abb343" exitCode=0 Jul 18 02:03:32.434676 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.434623 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" event={"ID":"de6109e1-4420-4de5-bff3-d3f957245170","Type":"ContainerDied","Data":"ca6616a6d16cec33307abbec1b9438a3655c52fbc56d3857ec1510fd95abb343"} Jul 18 02:03:32.436818 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.436799 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" Jul 18 02:03:32.436875 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.436808 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf" event={"ID":"18429789-5590-4420-99cb-ada89afc4685","Type":"ContainerDied","Data":"e14176b1c13335003b54cb89978b3f0b13e68d4f434c14d159b20c0ccf9714f6"} Jul 18 02:03:32.436875 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.436848 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e14176b1c13335003b54cb89978b3f0b13e68d4f434c14d159b20c0ccf9714f6" Jul 18 02:03:32.574399 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.574369 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:32.625505 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.625467 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-bundle\") pod \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " Jul 18 02:03:32.625505 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.625512 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-util\") pod \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " Jul 18 02:03:32.625766 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.625538 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwpjt\" (UniqueName: \"kubernetes.io/projected/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-kube-api-access-cwpjt\") pod \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\" (UID: \"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22\") " Jul 18 02:03:32.626184 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.626156 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-bundle" (OuterVolumeSpecName: "bundle") pod "b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" (UID: "b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:03:32.627623 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.627603 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-kube-api-access-cwpjt" (OuterVolumeSpecName: "kube-api-access-cwpjt") pod "b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" (UID: "b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22"). InnerVolumeSpecName "kube-api-access-cwpjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:03:32.630887 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.630847 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-util" (OuterVolumeSpecName: "util") pod "b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" (UID: "b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:03:32.726205 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.726166 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-bundle\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:32.726205 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.726201 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-util\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:32.726450 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:32.726218 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwpjt\" (UniqueName: \"kubernetes.io/projected/b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22-kube-api-access-cwpjt\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:33.443388 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.443353 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" event={"ID":"b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22","Type":"ContainerDied","Data":"f76f0c95d88e31876c9f6ddf4a8309bc073c5ce4c3dc1e2f7edabbff9c1a603e"} Jul 18 02:03:33.443864 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.443412 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f76f0c95d88e31876c9f6ddf4a8309bc073c5ce4c3dc1e2f7edabbff9c1a603e" Jul 18 02:03:33.443864 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.443366 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm" Jul 18 02:03:33.445557 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.445453 2573 generic.go:356] "Generic (PLEG): container finished" podID="ccbd3263-6251-4e1c-8fe7-713e2fc39d53" containerID="61bdecb14e73e928aeb4e4cdce4e0a547aebf6e8401aef48aef0ed62baf098b6" exitCode=0 Jul 18 02:03:33.445703 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.445560 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" event={"ID":"ccbd3263-6251-4e1c-8fe7-713e2fc39d53","Type":"ContainerDied","Data":"61bdecb14e73e928aeb4e4cdce4e0a547aebf6e8401aef48aef0ed62baf098b6"} Jul 18 02:03:33.579635 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.579611 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:33.633532 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.633490 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-util\") pod \"de6109e1-4420-4de5-bff3-d3f957245170\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " Jul 18 02:03:33.633532 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.633534 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-bundle\") pod \"de6109e1-4420-4de5-bff3-d3f957245170\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " Jul 18 02:03:33.633749 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.633552 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88qzt\" (UniqueName: \"kubernetes.io/projected/de6109e1-4420-4de5-bff3-d3f957245170-kube-api-access-88qzt\") pod \"de6109e1-4420-4de5-bff3-d3f957245170\" (UID: \"de6109e1-4420-4de5-bff3-d3f957245170\") " Jul 18 02:03:33.634103 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.634072 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-bundle" (OuterVolumeSpecName: "bundle") pod "de6109e1-4420-4de5-bff3-d3f957245170" (UID: "de6109e1-4420-4de5-bff3-d3f957245170"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:03:33.635677 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.635651 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de6109e1-4420-4de5-bff3-d3f957245170-kube-api-access-88qzt" (OuterVolumeSpecName: "kube-api-access-88qzt") pod "de6109e1-4420-4de5-bff3-d3f957245170" (UID: "de6109e1-4420-4de5-bff3-d3f957245170"). InnerVolumeSpecName "kube-api-access-88qzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:03:33.638923 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.638886 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-util" (OuterVolumeSpecName: "util") pod "de6109e1-4420-4de5-bff3-d3f957245170" (UID: "de6109e1-4420-4de5-bff3-d3f957245170"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:03:33.734106 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.734022 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-util\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:33.734106 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.734052 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de6109e1-4420-4de5-bff3-d3f957245170-bundle\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:33.734106 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:33.734062 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88qzt\" (UniqueName: \"kubernetes.io/projected/de6109e1-4420-4de5-bff3-d3f957245170-kube-api-access-88qzt\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:34.452013 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.451963 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" Jul 18 02:03:34.452013 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.452003 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h" event={"ID":"de6109e1-4420-4de5-bff3-d3f957245170","Type":"ContainerDied","Data":"b86bb7861a30b51e608df19c0d72d5f122b0dd765b28a34346409e559d481781"} Jul 18 02:03:34.452433 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.452050 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b86bb7861a30b51e608df19c0d72d5f122b0dd765b28a34346409e559d481781" Jul 18 02:03:34.581095 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.581070 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:34.641884 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.641841 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zz2kt\" (UniqueName: \"kubernetes.io/projected/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-kube-api-access-zz2kt\") pod \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " Jul 18 02:03:34.641884 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.641886 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-bundle\") pod \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " Jul 18 02:03:34.642146 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.642010 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-util\") pod \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\" (UID: \"ccbd3263-6251-4e1c-8fe7-713e2fc39d53\") " Jul 18 02:03:34.642476 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.642447 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-bundle" (OuterVolumeSpecName: "bundle") pod "ccbd3263-6251-4e1c-8fe7-713e2fc39d53" (UID: "ccbd3263-6251-4e1c-8fe7-713e2fc39d53"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:03:34.644014 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.643992 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-kube-api-access-zz2kt" (OuterVolumeSpecName: "kube-api-access-zz2kt") pod "ccbd3263-6251-4e1c-8fe7-713e2fc39d53" (UID: "ccbd3263-6251-4e1c-8fe7-713e2fc39d53"). InnerVolumeSpecName "kube-api-access-zz2kt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:03:34.646502 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.646481 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-util" (OuterVolumeSpecName: "util") pod "ccbd3263-6251-4e1c-8fe7-713e2fc39d53" (UID: "ccbd3263-6251-4e1c-8fe7-713e2fc39d53"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:03:34.743015 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.742956 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-util\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:34.743015 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.743017 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zz2kt\" (UniqueName: \"kubernetes.io/projected/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-kube-api-access-zz2kt\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:34.743212 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:34.743028 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ccbd3263-6251-4e1c-8fe7-713e2fc39d53-bundle\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:03:35.458683 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:35.458652 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" Jul 18 02:03:35.459099 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:35.458650 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc" event={"ID":"ccbd3263-6251-4e1c-8fe7-713e2fc39d53","Type":"ContainerDied","Data":"6a18c35a14c28ad761cd501a2318e32dd2a7b791d6f8f786acd469c6144be4ae"} Jul 18 02:03:35.459099 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:35.458770 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a18c35a14c28ad761cd501a2318e32dd2a7b791d6f8f786acd469c6144be4ae" Jul 18 02:03:40.508024 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:40.507958 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:43.145409 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145367 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c"] Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145682 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18429789-5590-4420-99cb-ada89afc4685" containerName="util" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145693 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18429789-5590-4420-99cb-ada89afc4685" containerName="util" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145701 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" containerName="util" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145707 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" containerName="util" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145719 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" containerName="extract" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145726 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" containerName="extract" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145732 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" containerName="pull" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145737 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" containerName="pull" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145747 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccbd3263-6251-4e1c-8fe7-713e2fc39d53" containerName="util" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145752 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccbd3263-6251-4e1c-8fe7-713e2fc39d53" containerName="util" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145757 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18429789-5590-4420-99cb-ada89afc4685" containerName="pull" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145761 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18429789-5590-4420-99cb-ada89afc4685" containerName="pull" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145769 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18429789-5590-4420-99cb-ada89afc4685" containerName="extract" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145774 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18429789-5590-4420-99cb-ada89afc4685" containerName="extract" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145779 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6109e1-4420-4de5-bff3-d3f957245170" containerName="util" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145784 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6109e1-4420-4de5-bff3-d3f957245170" containerName="util" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145789 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6109e1-4420-4de5-bff3-d3f957245170" containerName="pull" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145794 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6109e1-4420-4de5-bff3-d3f957245170" containerName="pull" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145798 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6109e1-4420-4de5-bff3-d3f957245170" containerName="extract" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145802 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6109e1-4420-4de5-bff3-d3f957245170" containerName="extract" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145807 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccbd3263-6251-4e1c-8fe7-713e2fc39d53" containerName="pull" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145811 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccbd3263-6251-4e1c-8fe7-713e2fc39d53" containerName="pull" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145818 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccbd3263-6251-4e1c-8fe7-713e2fc39d53" containerName="extract" Jul 18 02:03:43.145834 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145823 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccbd3263-6251-4e1c-8fe7-713e2fc39d53" containerName="extract" Jul 18 02:03:43.146514 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145866 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccbd3263-6251-4e1c-8fe7-713e2fc39d53" containerName="extract" Jul 18 02:03:43.146514 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145873 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="18429789-5590-4420-99cb-ada89afc4685" containerName="extract" Jul 18 02:03:43.146514 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145878 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22" containerName="extract" Jul 18 02:03:43.146514 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.145883 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="de6109e1-4420-4de5-bff3-d3f957245170" containerName="extract" Jul 18 02:03:43.147822 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.147804 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:03:43.150380 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.150349 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7w87n\"" Jul 18 02:03:43.164513 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.164481 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c"] Jul 18 02:03:43.217286 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.217242 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfdk2\" (UniqueName: \"kubernetes.io/projected/120ac105-143b-4ff1-9709-5206df00dbad-kube-api-access-qfdk2\") pod \"limitador-operator-controller-manager-859d7b78bd-n6q5c\" (UID: \"120ac105-143b-4ff1-9709-5206df00dbad\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:03:43.318132 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.318092 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfdk2\" (UniqueName: \"kubernetes.io/projected/120ac105-143b-4ff1-9709-5206df00dbad-kube-api-access-qfdk2\") pod \"limitador-operator-controller-manager-859d7b78bd-n6q5c\" (UID: \"120ac105-143b-4ff1-9709-5206df00dbad\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:03:43.327131 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.327098 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfdk2\" (UniqueName: \"kubernetes.io/projected/120ac105-143b-4ff1-9709-5206df00dbad-kube-api-access-qfdk2\") pod \"limitador-operator-controller-manager-859d7b78bd-n6q5c\" (UID: \"120ac105-143b-4ff1-9709-5206df00dbad\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:03:43.458249 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.458154 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:03:43.600751 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.600723 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c"] Jul 18 02:03:43.603352 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:03:43.603322 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod120ac105_143b_4ff1_9709_5206df00dbad.slice/crio-8de2a0836ed2c8faf10c9c69c53f9b821088e6654629fc47829687512758b924 WatchSource:0}: Error finding container 8de2a0836ed2c8faf10c9c69c53f9b821088e6654629fc47829687512758b924: Status 404 returned error can't find the container with id 8de2a0836ed2c8faf10c9c69c53f9b821088e6654629fc47829687512758b924 Jul 18 02:03:43.904874 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:43.904840 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:44.444863 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:44.444816 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:44.505329 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:44.505287 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" event={"ID":"120ac105-143b-4ff1-9709-5206df00dbad","Type":"ContainerStarted","Data":"8de2a0836ed2c8faf10c9c69c53f9b821088e6654629fc47829687512758b924"} Jul 18 02:03:44.546019 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:44.545945 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:44.699742 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:44.699644 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:45.511442 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:45.511403 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" event={"ID":"120ac105-143b-4ff1-9709-5206df00dbad","Type":"ContainerStarted","Data":"64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770"} Jul 18 02:03:45.511858 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:45.511552 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:03:45.531295 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:45.531227 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" podStartSLOduration=1.040564056 podStartE2EDuration="2.531211832s" totalImagesPullingTime="1.490647776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:03:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:03:45.53056124 +0000 UTC m=+413.318897824" watchObservedRunningTime="2026-07-18 02:03:45.531211832 +0000 UTC m=+413.319548414" Jul 18 02:03:47.294390 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:47.294353 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x"] Jul 18 02:03:47.296635 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:47.296617 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x" Jul 18 02:03:47.299025 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:47.299002 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5t4s9\"" Jul 18 02:03:47.308119 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:47.308091 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x"] Jul 18 02:03:47.356845 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:47.356802 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m68c6\" (UniqueName: \"kubernetes.io/projected/04dd269b-879b-4b6a-9ed8-6959092e82ee-kube-api-access-m68c6\") pod \"authorino-operator-75cc8d5cb7-zkb2x\" (UID: \"04dd269b-879b-4b6a-9ed8-6959092e82ee\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x" Jul 18 02:03:47.458190 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:47.458142 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m68c6\" (UniqueName: \"kubernetes.io/projected/04dd269b-879b-4b6a-9ed8-6959092e82ee-kube-api-access-m68c6\") pod \"authorino-operator-75cc8d5cb7-zkb2x\" (UID: \"04dd269b-879b-4b6a-9ed8-6959092e82ee\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x" Jul 18 02:03:47.466119 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:47.466084 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m68c6\" (UniqueName: \"kubernetes.io/projected/04dd269b-879b-4b6a-9ed8-6959092e82ee-kube-api-access-m68c6\") pod \"authorino-operator-75cc8d5cb7-zkb2x\" (UID: \"04dd269b-879b-4b6a-9ed8-6959092e82ee\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x" Jul 18 02:03:47.606649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:47.606547 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x" Jul 18 02:03:47.744344 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:47.744318 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x"] Jul 18 02:03:47.747107 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:03:47.747073 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04dd269b_879b_4b6a_9ed8_6959092e82ee.slice/crio-6884468f3a8e84999300be6835dcea27b57c5d07096bdcbeb836d18fe660d44f WatchSource:0}: Error finding container 6884468f3a8e84999300be6835dcea27b57c5d07096bdcbeb836d18fe660d44f: Status 404 returned error can't find the container with id 6884468f3a8e84999300be6835dcea27b57c5d07096bdcbeb836d18fe660d44f Jul 18 02:03:47.956814 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:47.956770 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:48.309366 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:48.309276 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:48.527909 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:48.527865 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x" event={"ID":"04dd269b-879b-4b6a-9ed8-6959092e82ee","Type":"ContainerStarted","Data":"6884468f3a8e84999300be6835dcea27b57c5d07096bdcbeb836d18fe660d44f"} Jul 18 02:03:50.138778 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:50.138744 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:50.548166 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:50.548130 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x" event={"ID":"04dd269b-879b-4b6a-9ed8-6959092e82ee","Type":"ContainerStarted","Data":"5ead5262693d06d6867f917bd23b8841fa3da72fef5ad8548d4224fe1cd55798"} Jul 18 02:03:50.548390 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:50.548320 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x" Jul 18 02:03:50.566446 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:50.566386 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x" podStartSLOduration=1.393664799 podStartE2EDuration="3.566368939s" totalImagesPullingTime="2.17270414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:03:50.565448693 +0000 UTC m=+418.353785276" watchObservedRunningTime="2026-07-18 02:03:50.566368939 +0000 UTC m=+418.354705521" Jul 18 02:03:51.142790 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:51.142758 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:51.288409 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:51.288374 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:52.945032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:52.944986 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:56.115510 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:56.115469 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:56.263943 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:56.263906 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:03:56.518062 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:03:56.518029 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:04:01.503073 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:01.503010 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:01.555906 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:01.555871 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zkb2x" Jul 18 02:04:01.583036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:01.582996 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:01.915245 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:01.915208 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:02.027186 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:02.027145 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:02.591837 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:02.591802 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:02.965835 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:02.965788 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:05.720282 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:05.720246 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:05.830945 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:05.830911 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:09.141850 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:09.141815 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:09.334037 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:09.333998 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:12.662016 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.661965 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c"] Jul 18 02:04:12.662401 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.662229 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" podUID="120ac105-143b-4ff1-9709-5206df00dbad" containerName="manager" containerID="cri-o://64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770" gracePeriod=2 Jul 18 02:04:12.679638 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.679608 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c"] Jul 18 02:04:12.682987 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:04:12.682940 2573 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"120ac105-143b-4ff1-9709-5206df00dbad\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-n6q5c\": pods \"limitador-operator-controller-manager-859d7b78bd-n6q5c\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:04:12.689842 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.689815 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf"] Jul 18 02:04:12.690148 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.690134 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="120ac105-143b-4ff1-9709-5206df00dbad" containerName="manager" Jul 18 02:04:12.690207 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.690150 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="120ac105-143b-4ff1-9709-5206df00dbad" containerName="manager" Jul 18 02:04:12.690258 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.690229 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="120ac105-143b-4ff1-9709-5206df00dbad" containerName="manager" Jul 18 02:04:12.692123 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.692106 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf" Jul 18 02:04:12.695334 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:04:12.695305 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-n6q5c\" is forbidden: User \"system:node:ip-10-0-130-187.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-187.ec2.internal' and this object" podUID="120ac105-143b-4ff1-9709-5206df00dbad" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:04:12.705466 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.705428 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf"] Jul 18 02:04:12.722749 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:04:12.722713 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-n6q5c\" is forbidden: User \"system:node:ip-10-0-130-187.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-187.ec2.internal' and this object" podUID="120ac105-143b-4ff1-9709-5206df00dbad" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:04:12.873285 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.873254 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4tcv\" (UniqueName: \"kubernetes.io/projected/394487e0-7e76-4cd7-837b-c31265df51c4-kube-api-access-k4tcv\") pod \"limitador-operator-controller-manager-859d7b78bd-9sntf\" (UID: \"394487e0-7e76-4cd7-837b-c31265df51c4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf" Jul 18 02:04:12.923145 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.923092 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:04:12.973821 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.973787 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4tcv\" (UniqueName: \"kubernetes.io/projected/394487e0-7e76-4cd7-837b-c31265df51c4-kube-api-access-k4tcv\") pod \"limitador-operator-controller-manager-859d7b78bd-9sntf\" (UID: \"394487e0-7e76-4cd7-837b-c31265df51c4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf" Jul 18 02:04:12.982299 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:12.982267 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4tcv\" (UniqueName: \"kubernetes.io/projected/394487e0-7e76-4cd7-837b-c31265df51c4-kube-api-access-k4tcv\") pod \"limitador-operator-controller-manager-859d7b78bd-9sntf\" (UID: \"394487e0-7e76-4cd7-837b-c31265df51c4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf" Jul 18 02:04:13.051477 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.051431 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf" Jul 18 02:04:13.075008 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.074954 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfdk2\" (UniqueName: \"kubernetes.io/projected/120ac105-143b-4ff1-9709-5206df00dbad-kube-api-access-qfdk2\") pod \"120ac105-143b-4ff1-9709-5206df00dbad\" (UID: \"120ac105-143b-4ff1-9709-5206df00dbad\") " Jul 18 02:04:13.077045 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.077011 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/120ac105-143b-4ff1-9709-5206df00dbad-kube-api-access-qfdk2" (OuterVolumeSpecName: "kube-api-access-qfdk2") pod "120ac105-143b-4ff1-9709-5206df00dbad" (UID: "120ac105-143b-4ff1-9709-5206df00dbad"). InnerVolumeSpecName "kube-api-access-qfdk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:04:13.175929 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.175852 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfdk2\" (UniqueName: \"kubernetes.io/projected/120ac105-143b-4ff1-9709-5206df00dbad-kube-api-access-qfdk2\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:04:13.192147 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.192110 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf"] Jul 18 02:04:13.195226 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:04:13.195198 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod394487e0_7e76_4cd7_837b_c31265df51c4.slice/crio-d55f211236d344452a4323d66c94d11e80be6b62e1402496e91a9801fb53c344 WatchSource:0}: Error finding container d55f211236d344452a4323d66c94d11e80be6b62e1402496e91a9801fb53c344: Status 404 returned error can't find the container with id d55f211236d344452a4323d66c94d11e80be6b62e1402496e91a9801fb53c344 Jul 18 02:04:13.655651 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.655616 2573 generic.go:356] "Generic (PLEG): container finished" podID="120ac105-143b-4ff1-9709-5206df00dbad" containerID="64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770" exitCode=0 Jul 18 02:04:13.655844 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.655694 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n6q5c" Jul 18 02:04:13.655844 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.655721 2573 scope.go:117] "RemoveContainer" containerID="64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770" Jul 18 02:04:13.657580 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.657555 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf" event={"ID":"394487e0-7e76-4cd7-837b-c31265df51c4","Type":"ContainerStarted","Data":"45ca742c40f6c01abb33cb6683f1967d587b36917f88dcde20a85512c5f38086"} Jul 18 02:04:13.657713 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.657640 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf" Jul 18 02:04:13.657713 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.657658 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf" event={"ID":"394487e0-7e76-4cd7-837b-c31265df51c4","Type":"ContainerStarted","Data":"d55f211236d344452a4323d66c94d11e80be6b62e1402496e91a9801fb53c344"} Jul 18 02:04:13.664736 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.664593 2573 scope.go:117] "RemoveContainer" containerID="64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770" Jul 18 02:04:13.664934 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:04:13.664868 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770\": container with ID starting with 64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770 not found: ID does not exist" containerID="64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770" Jul 18 02:04:13.664934 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.664892 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770"} err="failed to get container status \"64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770\": rpc error: code = NotFound desc = could not find container \"64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770\": container with ID starting with 64dc26eb903fcdebb253b1e43ad56904ec54b4c769ca844013199efd2519c770 not found: ID does not exist" Jul 18 02:04:13.687250 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:13.687201 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf" podStartSLOduration=1.687184848 podStartE2EDuration="1.687184848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:04:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:04:13.684869703 +0000 UTC m=+441.473206289" watchObservedRunningTime="2026-07-18 02:04:13.687184848 +0000 UTC m=+441.475521431" Jul 18 02:04:14.721892 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:14.721854 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="120ac105-143b-4ff1-9709-5206df00dbad" path="/var/lib/kubelet/pods/120ac105-143b-4ff1-9709-5206df00dbad/volumes" Jul 18 02:04:15.765341 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:15.765249 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:15.871556 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:15.871514 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:19.557316 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:19.557280 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:19.657687 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:19.657652 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:19.757327 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:19.757288 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:19.819388 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:19.819292 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:23.355336 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:23.355288 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:23.408524 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:23.408488 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:24.665888 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:24.665855 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9sntf" Jul 18 02:04:29.959502 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:29.959466 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:30.061406 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:30.061367 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:30.123067 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:30.123026 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:49.712510 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:49.712470 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:49.813703 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:49.813663 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:57.527285 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.527247 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886"] Jul 18 02:04:57.531510 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.531493 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.533856 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.533834 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-pshpc\"" Jul 18 02:04:57.540642 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.540616 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886"] Jul 18 02:04:57.568955 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.568907 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:57.614206 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.614164 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.614382 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.614211 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.614382 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.614284 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33381711-7e71-4b61-bf6a-e101f47a717b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.614382 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.614356 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33381711-7e71-4b61-bf6a-e101f47a717b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.614507 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.614405 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.614507 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.614436 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33381711-7e71-4b61-bf6a-e101f47a717b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.614507 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.614474 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mwc5\" (UniqueName: \"kubernetes.io/projected/33381711-7e71-4b61-bf6a-e101f47a717b-kube-api-access-8mwc5\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.614599 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.614533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.614599 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.614581 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.715775 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.715740 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.715962 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.715812 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.715962 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.715838 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.715962 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.715855 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.715962 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.715887 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33381711-7e71-4b61-bf6a-e101f47a717b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.715962 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.715913 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33381711-7e71-4b61-bf6a-e101f47a717b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.715962 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.715934 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.716217 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.715963 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33381711-7e71-4b61-bf6a-e101f47a717b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.716217 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.716085 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mwc5\" (UniqueName: \"kubernetes.io/projected/33381711-7e71-4b61-bf6a-e101f47a717b-kube-api-access-8mwc5\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.716284 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.716222 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.716578 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.716554 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.716663 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.716554 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33381711-7e71-4b61-bf6a-e101f47a717b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.716663 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.716618 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.716771 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.716687 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.718323 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.718303 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33381711-7e71-4b61-bf6a-e101f47a717b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.718555 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.718539 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33381711-7e71-4b61-bf6a-e101f47a717b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.723818 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.723790 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33381711-7e71-4b61-bf6a-e101f47a717b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.723995 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.723960 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mwc5\" (UniqueName: \"kubernetes.io/projected/33381711-7e71-4b61-bf6a-e101f47a717b-kube-api-access-8mwc5\") pod \"maas-default-gateway-openshift-default-56654dcf7-zt886\" (UID: \"33381711-7e71-4b61-bf6a-e101f47a717b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.768216 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.768174 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:04:57.841831 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.841733 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:57.985735 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.985703 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886"] Jul 18 02:04:57.989581 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:04:57.989555 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33381711_7e71_4b61_bf6a_e101f47a717b.slice/crio-2325b28ba8fccb5ed82e1f6d4b79144e9fb17a9961d3fd33dcefb132fb837416 WatchSource:0}: Error finding container 2325b28ba8fccb5ed82e1f6d4b79144e9fb17a9961d3fd33dcefb132fb837416: Status 404 returned error can't find the container with id 2325b28ba8fccb5ed82e1f6d4b79144e9fb17a9961d3fd33dcefb132fb837416 Jul 18 02:04:57.991517 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.991486 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:04:57.991577 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.991550 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:04:57.991617 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:57.991579 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:04:58.869854 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:58.869813 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" event={"ID":"33381711-7e71-4b61-bf6a-e101f47a717b","Type":"ContainerStarted","Data":"7258e6b3b21108400463368b892ee21247afab3ccc987acd79fe5320a85014ba"} Jul 18 02:04:58.870291 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:58.869875 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" event={"ID":"33381711-7e71-4b61-bf6a-e101f47a717b","Type":"ContainerStarted","Data":"2325b28ba8fccb5ed82e1f6d4b79144e9fb17a9961d3fd33dcefb132fb837416"} Jul 18 02:04:58.890689 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:58.890635 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" podStartSLOduration=1.890616041 podStartE2EDuration="1.890616041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:04:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:04:58.889153644 +0000 UTC m=+486.677490254" watchObservedRunningTime="2026-07-18 02:04:58.890616041 +0000 UTC m=+486.678952624" Jul 18 02:04:59.842849 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:59.842807 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:59.847728 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:59.847700 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:59.874790 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:59.874758 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:04:59.875763 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:04:59.875745 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:05:00.308683 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:00.308647 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:00.404965 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:00.404931 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:00.506287 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:00.506243 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:01.118244 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.118208 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-stjgl"] Jul 18 02:05:01.121600 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.121577 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-stjgl" Jul 18 02:05:01.124405 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.124378 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hxsnd\"" Jul 18 02:05:01.126802 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.126774 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-stjgl"] Jul 18 02:05:01.154553 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.154508 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:01.250591 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.250549 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p5kc\" (UniqueName: \"kubernetes.io/projected/52c766ec-a383-48be-a842-a0d458572b4b-kube-api-access-8p5kc\") pod \"authorino-58cd955675-stjgl\" (UID: \"52c766ec-a383-48be-a842-a0d458572b4b\") " pod="kuadrant-system/authorino-58cd955675-stjgl" Jul 18 02:05:01.312463 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.312425 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:01.351238 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.351198 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p5kc\" (UniqueName: \"kubernetes.io/projected/52c766ec-a383-48be-a842-a0d458572b4b-kube-api-access-8p5kc\") pod \"authorino-58cd955675-stjgl\" (UID: \"52c766ec-a383-48be-a842-a0d458572b4b\") " pod="kuadrant-system/authorino-58cd955675-stjgl" Jul 18 02:05:01.361986 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.361937 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p5kc\" (UniqueName: \"kubernetes.io/projected/52c766ec-a383-48be-a842-a0d458572b4b-kube-api-access-8p5kc\") pod \"authorino-58cd955675-stjgl\" (UID: \"52c766ec-a383-48be-a842-a0d458572b4b\") " pod="kuadrant-system/authorino-58cd955675-stjgl" Jul 18 02:05:01.433869 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.433830 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-stjgl" Jul 18 02:05:01.567784 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.567753 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-stjgl"] Jul 18 02:05:01.569697 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:05:01.569670 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52c766ec_a383_48be_a842_a0d458572b4b.slice/crio-715ad1d2d6bcb1ecef2cc555c25446891ae29e88ec29cf868cea56bee3927413 WatchSource:0}: Error finding container 715ad1d2d6bcb1ecef2cc555c25446891ae29e88ec29cf868cea56bee3927413: Status 404 returned error can't find the container with id 715ad1d2d6bcb1ecef2cc555c25446891ae29e88ec29cf868cea56bee3927413 Jul 18 02:05:01.885024 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:01.884963 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-stjgl" event={"ID":"52c766ec-a383-48be-a842-a0d458572b4b","Type":"ContainerStarted","Data":"715ad1d2d6bcb1ecef2cc555c25446891ae29e88ec29cf868cea56bee3927413"} Jul 18 02:05:04.906205 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:04.906141 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-stjgl" event={"ID":"52c766ec-a383-48be-a842-a0d458572b4b","Type":"ContainerStarted","Data":"1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5"} Jul 18 02:05:04.921399 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:04.921320 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-stjgl" podStartSLOduration=0.688131944 podStartE2EDuration="3.921303234s" totalImagesPullingTime="3.23317129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:05:04.92052938 +0000 UTC m=+492.708865965" watchObservedRunningTime="2026-07-18 02:05:04.921303234 +0000 UTC m=+492.709639817" Jul 18 02:05:05.588893 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:05.588857 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:07.455729 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.455687 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-stjgl"] Jul 18 02:05:07.456164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.455870 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-stjgl" podUID="52c766ec-a383-48be-a842-a0d458572b4b" containerName="authorino" containerID="cri-o://1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5" gracePeriod=30 Jul 18 02:05:07.716005 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.715913 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-stjgl" Jul 18 02:05:07.806950 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.806911 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p5kc\" (UniqueName: \"kubernetes.io/projected/52c766ec-a383-48be-a842-a0d458572b4b-kube-api-access-8p5kc\") pod \"52c766ec-a383-48be-a842-a0d458572b4b\" (UID: \"52c766ec-a383-48be-a842-a0d458572b4b\") " Jul 18 02:05:07.809099 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.809063 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c766ec-a383-48be-a842-a0d458572b4b-kube-api-access-8p5kc" (OuterVolumeSpecName: "kube-api-access-8p5kc") pod "52c766ec-a383-48be-a842-a0d458572b4b" (UID: "52c766ec-a383-48be-a842-a0d458572b4b"). InnerVolumeSpecName "kube-api-access-8p5kc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:05:07.907588 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.907550 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p5kc\" (UniqueName: \"kubernetes.io/projected/52c766ec-a383-48be-a842-a0d458572b4b-kube-api-access-8p5kc\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:05:07.921850 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.921815 2573 generic.go:356] "Generic (PLEG): container finished" podID="52c766ec-a383-48be-a842-a0d458572b4b" containerID="1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5" exitCode=0 Jul 18 02:05:07.922036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.921865 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-stjgl" Jul 18 02:05:07.922036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.921904 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-stjgl" event={"ID":"52c766ec-a383-48be-a842-a0d458572b4b","Type":"ContainerDied","Data":"1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5"} Jul 18 02:05:07.922036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.921950 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-stjgl" event={"ID":"52c766ec-a383-48be-a842-a0d458572b4b","Type":"ContainerDied","Data":"715ad1d2d6bcb1ecef2cc555c25446891ae29e88ec29cf868cea56bee3927413"} Jul 18 02:05:07.922036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.921980 2573 scope.go:117] "RemoveContainer" containerID="1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5" Jul 18 02:05:07.931208 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.931185 2573 scope.go:117] "RemoveContainer" containerID="1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5" Jul 18 02:05:07.931525 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:05:07.931503 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5\": container with ID starting with 1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5 not found: ID does not exist" containerID="1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5" Jul 18 02:05:07.931592 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.931533 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5"} err="failed to get container status \"1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5\": rpc error: code = NotFound desc = could not find container \"1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5\": container with ID starting with 1dbc73e61b37ee59f3284d3d2c08792421e3fbe97b4b9ca5ea22dccfc1a237b5 not found: ID does not exist" Jul 18 02:05:07.945391 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.945357 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-stjgl"] Jul 18 02:05:07.947204 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:07.947178 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-stjgl"] Jul 18 02:05:08.721883 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:08.721849 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52c766ec-a383-48be-a842-a0d458572b4b" path="/var/lib/kubelet/pods/52c766ec-a383-48be-a842-a0d458572b4b/volumes" Jul 18 02:05:21.123040 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:21.123005 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:21.222425 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:21.222391 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:21.639920 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:21.639890 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:34.769455 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:34.769405 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:34.933840 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:34.933804 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:35.007427 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.007387 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qc8bs"] Jul 18 02:05:35.007733 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.007721 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c766ec-a383-48be-a842-a0d458572b4b" containerName="authorino" Jul 18 02:05:35.007776 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.007735 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c766ec-a383-48be-a842-a0d458572b4b" containerName="authorino" Jul 18 02:05:35.007810 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.007786 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c766ec-a383-48be-a842-a0d458572b4b" containerName="authorino" Jul 18 02:05:35.014031 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.014007 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qc8bs" Jul 18 02:05:35.016434 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.016409 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hxsnd\"" Jul 18 02:05:35.020439 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.020386 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qc8bs"] Jul 18 02:05:35.035555 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.035522 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5qll\" (UniqueName: \"kubernetes.io/projected/04882795-48b0-4b40-bcd0-841aa0fab4f9-kube-api-access-x5qll\") pod \"authorino-5d4b67c446-qc8bs\" (UID: \"04882795-48b0-4b40-bcd0-841aa0fab4f9\") " pod="kuadrant-system/authorino-5d4b67c446-qc8bs" Jul 18 02:05:35.136190 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.136145 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5qll\" (UniqueName: \"kubernetes.io/projected/04882795-48b0-4b40-bcd0-841aa0fab4f9-kube-api-access-x5qll\") pod \"authorino-5d4b67c446-qc8bs\" (UID: \"04882795-48b0-4b40-bcd0-841aa0fab4f9\") " pod="kuadrant-system/authorino-5d4b67c446-qc8bs" Jul 18 02:05:35.143986 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.143935 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5qll\" (UniqueName: \"kubernetes.io/projected/04882795-48b0-4b40-bcd0-841aa0fab4f9-kube-api-access-x5qll\") pod \"authorino-5d4b67c446-qc8bs\" (UID: \"04882795-48b0-4b40-bcd0-841aa0fab4f9\") " pod="kuadrant-system/authorino-5d4b67c446-qc8bs" Jul 18 02:05:35.240038 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.240002 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qc8bs"] Jul 18 02:05:35.240240 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.240227 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qc8bs" Jul 18 02:05:35.266987 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.266935 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7898cf6c9b-fmjtc"] Jul 18 02:05:35.272181 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.272120 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7898cf6c9b-fmjtc" Jul 18 02:05:35.276751 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.276720 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7898cf6c9b-fmjtc"] Jul 18 02:05:35.337714 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.337666 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r9cc\" (UniqueName: \"kubernetes.io/projected/33e8bc55-b719-4c02-b69b-080e35bd9d43-kube-api-access-9r9cc\") pod \"authorino-7898cf6c9b-fmjtc\" (UID: \"33e8bc55-b719-4c02-b69b-080e35bd9d43\") " pod="kuadrant-system/authorino-7898cf6c9b-fmjtc" Jul 18 02:05:35.342824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.342787 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7898cf6c9b-fmjtc"] Jul 18 02:05:35.343098 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:05:35.343075 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9r9cc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7898cf6c9b-fmjtc" podUID="33e8bc55-b719-4c02-b69b-080e35bd9d43" Jul 18 02:05:35.371811 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.371775 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6859878464-pcp6z"] Jul 18 02:05:35.377904 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.377878 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6859878464-pcp6z" Jul 18 02:05:35.380503 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.380483 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 18 02:05:35.385061 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.385036 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6859878464-pcp6z"] Jul 18 02:05:35.387506 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.387483 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qc8bs"] Jul 18 02:05:35.388580 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:05:35.388546 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04882795_48b0_4b40_bcd0_841aa0fab4f9.slice/crio-d4f1e79f36c9e03af1acae7314e4d45382ed8d796b292b29cf4cdeb45ba788c2 WatchSource:0}: Error finding container d4f1e79f36c9e03af1acae7314e4d45382ed8d796b292b29cf4cdeb45ba788c2: Status 404 returned error can't find the container with id d4f1e79f36c9e03af1acae7314e4d45382ed8d796b292b29cf4cdeb45ba788c2 Jul 18 02:05:35.438689 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.438645 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-tls-cert\") pod \"authorino-6859878464-pcp6z\" (UID: \"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c\") " pod="kuadrant-system/authorino-6859878464-pcp6z" Jul 18 02:05:35.438886 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.438698 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m42tt\" (UniqueName: \"kubernetes.io/projected/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-kube-api-access-m42tt\") pod \"authorino-6859878464-pcp6z\" (UID: \"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c\") " pod="kuadrant-system/authorino-6859878464-pcp6z" Jul 18 02:05:35.438886 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.438759 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r9cc\" (UniqueName: \"kubernetes.io/projected/33e8bc55-b719-4c02-b69b-080e35bd9d43-kube-api-access-9r9cc\") pod \"authorino-7898cf6c9b-fmjtc\" (UID: \"33e8bc55-b719-4c02-b69b-080e35bd9d43\") " pod="kuadrant-system/authorino-7898cf6c9b-fmjtc" Jul 18 02:05:35.447901 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.447867 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r9cc\" (UniqueName: \"kubernetes.io/projected/33e8bc55-b719-4c02-b69b-080e35bd9d43-kube-api-access-9r9cc\") pod \"authorino-7898cf6c9b-fmjtc\" (UID: \"33e8bc55-b719-4c02-b69b-080e35bd9d43\") " pod="kuadrant-system/authorino-7898cf6c9b-fmjtc" Jul 18 02:05:35.539325 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.539235 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-tls-cert\") pod \"authorino-6859878464-pcp6z\" (UID: \"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c\") " pod="kuadrant-system/authorino-6859878464-pcp6z" Jul 18 02:05:35.539325 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.539279 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m42tt\" (UniqueName: \"kubernetes.io/projected/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-kube-api-access-m42tt\") pod \"authorino-6859878464-pcp6z\" (UID: \"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c\") " pod="kuadrant-system/authorino-6859878464-pcp6z" Jul 18 02:05:35.541735 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.541708 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-tls-cert\") pod \"authorino-6859878464-pcp6z\" (UID: \"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c\") " pod="kuadrant-system/authorino-6859878464-pcp6z" Jul 18 02:05:35.546770 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.546744 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m42tt\" (UniqueName: \"kubernetes.io/projected/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-kube-api-access-m42tt\") pod \"authorino-6859878464-pcp6z\" (UID: \"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c\") " pod="kuadrant-system/authorino-6859878464-pcp6z" Jul 18 02:05:35.689595 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.689561 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6859878464-pcp6z" Jul 18 02:05:35.849957 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:35.849877 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6859878464-pcp6z"] Jul 18 02:05:35.852284 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:05:35.852257 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0817f2bc_39a2_47cf_8ff2_2d2f194ba30c.slice/crio-2792d77149f904989039039abeb0e7bca942d70ba50e3eabbc44e3996ef9ad5a WatchSource:0}: Error finding container 2792d77149f904989039039abeb0e7bca942d70ba50e3eabbc44e3996ef9ad5a: Status 404 returned error can't find the container with id 2792d77149f904989039039abeb0e7bca942d70ba50e3eabbc44e3996ef9ad5a Jul 18 02:05:36.053760 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:36.053676 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6859878464-pcp6z" event={"ID":"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c","Type":"ContainerStarted","Data":"2792d77149f904989039039abeb0e7bca942d70ba50e3eabbc44e3996ef9ad5a"} Jul 18 02:05:36.054718 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:36.054696 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7898cf6c9b-fmjtc" Jul 18 02:05:36.054841 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:36.054730 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-qc8bs" event={"ID":"04882795-48b0-4b40-bcd0-841aa0fab4f9","Type":"ContainerStarted","Data":"d4f1e79f36c9e03af1acae7314e4d45382ed8d796b292b29cf4cdeb45ba788c2"} Jul 18 02:05:36.071567 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:36.071537 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7898cf6c9b-fmjtc" Jul 18 02:05:36.144380 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:36.144346 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r9cc\" (UniqueName: \"kubernetes.io/projected/33e8bc55-b719-4c02-b69b-080e35bd9d43-kube-api-access-9r9cc\") pod \"33e8bc55-b719-4c02-b69b-080e35bd9d43\" (UID: \"33e8bc55-b719-4c02-b69b-080e35bd9d43\") " Jul 18 02:05:36.146457 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:36.146434 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e8bc55-b719-4c02-b69b-080e35bd9d43-kube-api-access-9r9cc" (OuterVolumeSpecName: "kube-api-access-9r9cc") pod "33e8bc55-b719-4c02-b69b-080e35bd9d43" (UID: "33e8bc55-b719-4c02-b69b-080e35bd9d43"). InnerVolumeSpecName "kube-api-access-9r9cc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:05:36.244933 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:36.244899 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r9cc\" (UniqueName: \"kubernetes.io/projected/33e8bc55-b719-4c02-b69b-080e35bd9d43-kube-api-access-9r9cc\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:05:37.061605 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.061497 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6859878464-pcp6z" event={"ID":"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c","Type":"ContainerStarted","Data":"c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b"} Jul 18 02:05:37.062994 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.062955 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-qc8bs" event={"ID":"04882795-48b0-4b40-bcd0-841aa0fab4f9","Type":"ContainerStarted","Data":"9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31"} Jul 18 02:05:37.063088 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.063034 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-qc8bs" podUID="04882795-48b0-4b40-bcd0-841aa0fab4f9" containerName="authorino" containerID="cri-o://9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31" gracePeriod=30 Jul 18 02:05:37.063088 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.063067 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7898cf6c9b-fmjtc" Jul 18 02:05:37.077229 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.077160 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6859878464-pcp6z" podStartSLOduration=1.6845205939999999 podStartE2EDuration="2.077143816s" totalImagesPullingTime="392.623222ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:05:37.076006697 +0000 UTC m=+524.864343277" watchObservedRunningTime="2026-07-18 02:05:37.077143816 +0000 UTC m=+524.865480399" Jul 18 02:05:37.104192 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.104158 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7898cf6c9b-fmjtc"] Jul 18 02:05:37.107306 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.107275 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7898cf6c9b-fmjtc"] Jul 18 02:05:37.120610 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.120562 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-qc8bs" podStartSLOduration=2.420991561 podStartE2EDuration="3.120545198s" totalImagesPullingTime="699.553637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:05:37.119712536 +0000 UTC m=+524.908049118" watchObservedRunningTime="2026-07-18 02:05:37.120545198 +0000 UTC m=+524.908881781" Jul 18 02:05:37.303583 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.303553 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qc8bs" Jul 18 02:05:37.354464 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.354385 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5qll\" (UniqueName: \"kubernetes.io/projected/04882795-48b0-4b40-bcd0-841aa0fab4f9-kube-api-access-x5qll\") pod \"04882795-48b0-4b40-bcd0-841aa0fab4f9\" (UID: \"04882795-48b0-4b40-bcd0-841aa0fab4f9\") " Jul 18 02:05:37.356490 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.356466 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04882795-48b0-4b40-bcd0-841aa0fab4f9-kube-api-access-x5qll" (OuterVolumeSpecName: "kube-api-access-x5qll") pod "04882795-48b0-4b40-bcd0-841aa0fab4f9" (UID: "04882795-48b0-4b40-bcd0-841aa0fab4f9"). InnerVolumeSpecName "kube-api-access-x5qll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:05:37.455309 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:37.455259 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5qll\" (UniqueName: \"kubernetes.io/projected/04882795-48b0-4b40-bcd0-841aa0fab4f9-kube-api-access-x5qll\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:05:38.075188 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.075150 2573 generic.go:356] "Generic (PLEG): container finished" podID="04882795-48b0-4b40-bcd0-841aa0fab4f9" containerID="9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31" exitCode=0 Jul 18 02:05:38.075601 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.075205 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qc8bs" Jul 18 02:05:38.075601 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.075242 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-qc8bs" event={"ID":"04882795-48b0-4b40-bcd0-841aa0fab4f9","Type":"ContainerDied","Data":"9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31"} Jul 18 02:05:38.075601 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.075296 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-qc8bs" event={"ID":"04882795-48b0-4b40-bcd0-841aa0fab4f9","Type":"ContainerDied","Data":"d4f1e79f36c9e03af1acae7314e4d45382ed8d796b292b29cf4cdeb45ba788c2"} Jul 18 02:05:38.075601 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.075322 2573 scope.go:117] "RemoveContainer" containerID="9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31" Jul 18 02:05:38.083670 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.083645 2573 scope.go:117] "RemoveContainer" containerID="9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31" Jul 18 02:05:38.083981 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:05:38.083948 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31\": container with ID starting with 9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31 not found: ID does not exist" containerID="9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31" Jul 18 02:05:38.084076 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.084057 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31"} err="failed to get container status \"9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31\": rpc error: code = NotFound desc = could not find container \"9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31\": container with ID starting with 9f6ad568d63823a9dfc423e74d4b9ff9693cdaa3fb7a5f5ba6cae0ee12338f31 not found: ID does not exist" Jul 18 02:05:38.095511 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.095476 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qc8bs"] Jul 18 02:05:38.099061 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.099032 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qc8bs"] Jul 18 02:05:38.722682 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.722638 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04882795-48b0-4b40-bcd0-841aa0fab4f9" path="/var/lib/kubelet/pods/04882795-48b0-4b40-bcd0-841aa0fab4f9/volumes" Jul 18 02:05:38.723063 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:38.723043 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e8bc55-b719-4c02-b69b-080e35bd9d43" path="/var/lib/kubelet/pods/33e8bc55-b719-4c02-b69b-080e35bd9d43/volumes" Jul 18 02:05:41.155985 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:41.155891 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:41.209909 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:41.209873 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:41.714779 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:41.714740 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:41.777358 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:41.777113 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:41.828572 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:41.828536 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:42.211448 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:42.211416 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:45.561435 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:45.561341 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:05:45.662116 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:05:45.662080 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:17.169774 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:17.169742 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:17.268220 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:17.268182 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:17.810126 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:17.810093 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:18.309307 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.309264 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:18.418136 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.418100 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:18.443082 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.443048 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-986hh"] Jul 18 02:06:18.443356 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.443345 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04882795-48b0-4b40-bcd0-841aa0fab4f9" containerName="authorino" Jul 18 02:06:18.443400 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.443358 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04882795-48b0-4b40-bcd0-841aa0fab4f9" containerName="authorino" Jul 18 02:06:18.443435 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.443412 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="04882795-48b0-4b40-bcd0-841aa0fab4f9" containerName="authorino" Jul 18 02:06:18.446152 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.446135 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-986hh" Jul 18 02:06:18.448471 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.448446 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 18 02:06:18.448622 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.448587 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dgccx\"" Jul 18 02:06:18.456055 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.456030 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-986hh"] Jul 18 02:06:18.600397 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.600311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls5vn\" (UniqueName: \"kubernetes.io/projected/be269f87-e314-4ca5-bcf4-24658f1a4dee-kube-api-access-ls5vn\") pod \"payload-processing-5bf47cd988-986hh\" (UID: \"be269f87-e314-4ca5-bcf4-24658f1a4dee\") " pod="openshift-ingress/payload-processing-5bf47cd988-986hh" Jul 18 02:06:18.600540 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.600452 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be269f87-e314-4ca5-bcf4-24658f1a4dee-plugins-config-volume\") pod \"payload-processing-5bf47cd988-986hh\" (UID: \"be269f87-e314-4ca5-bcf4-24658f1a4dee\") " pod="openshift-ingress/payload-processing-5bf47cd988-986hh" Jul 18 02:06:18.701127 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.701085 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls5vn\" (UniqueName: \"kubernetes.io/projected/be269f87-e314-4ca5-bcf4-24658f1a4dee-kube-api-access-ls5vn\") pod \"payload-processing-5bf47cd988-986hh\" (UID: \"be269f87-e314-4ca5-bcf4-24658f1a4dee\") " pod="openshift-ingress/payload-processing-5bf47cd988-986hh" Jul 18 02:06:18.701322 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.701176 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be269f87-e314-4ca5-bcf4-24658f1a4dee-plugins-config-volume\") pod \"payload-processing-5bf47cd988-986hh\" (UID: \"be269f87-e314-4ca5-bcf4-24658f1a4dee\") " pod="openshift-ingress/payload-processing-5bf47cd988-986hh" Jul 18 02:06:18.701920 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.701899 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be269f87-e314-4ca5-bcf4-24658f1a4dee-plugins-config-volume\") pod \"payload-processing-5bf47cd988-986hh\" (UID: \"be269f87-e314-4ca5-bcf4-24658f1a4dee\") " pod="openshift-ingress/payload-processing-5bf47cd988-986hh" Jul 18 02:06:18.709079 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.709059 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls5vn\" (UniqueName: \"kubernetes.io/projected/be269f87-e314-4ca5-bcf4-24658f1a4dee-kube-api-access-ls5vn\") pod \"payload-processing-5bf47cd988-986hh\" (UID: \"be269f87-e314-4ca5-bcf4-24658f1a4dee\") " pod="openshift-ingress/payload-processing-5bf47cd988-986hh" Jul 18 02:06:18.711540 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.711506 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:18.758892 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.758863 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-986hh" Jul 18 02:06:18.895213 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:18.895184 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-986hh"] Jul 18 02:06:18.897314 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:06:18.897288 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe269f87_e314_4ca5_bcf4_24658f1a4dee.slice/crio-052b88275f62aa6600d474622c6d27f178950d122ebe47c4607b89603451db17 WatchSource:0}: Error finding container 052b88275f62aa6600d474622c6d27f178950d122ebe47c4607b89603451db17: Status 404 returned error can't find the container with id 052b88275f62aa6600d474622c6d27f178950d122ebe47c4607b89603451db17 Jul 18 02:06:19.272477 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:19.272391 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-986hh" event={"ID":"be269f87-e314-4ca5-bcf4-24658f1a4dee","Type":"ContainerStarted","Data":"052b88275f62aa6600d474622c6d27f178950d122ebe47c4607b89603451db17"} Jul 18 02:06:21.311278 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:21.307941 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-986hh" event={"ID":"be269f87-e314-4ca5-bcf4-24658f1a4dee","Type":"ContainerStarted","Data":"1af475cfe7f87794a2a82dec9d1747815487ab8495228f2d80ead5fe377221dd"} Jul 18 02:06:21.311278 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:21.308769 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-986hh" Jul 18 02:06:21.325344 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:21.325295 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-986hh" podStartSLOduration=1.490927208 podStartE2EDuration="3.325281908s" totalImagesPullingTime="1.8343547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:06:21.323406426 +0000 UTC m=+569.111743006" watchObservedRunningTime="2026-07-18 02:06:21.325281908 +0000 UTC m=+569.113618490" Jul 18 02:06:26.335712 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:26.335683 2573 generic.go:356] "Generic (PLEG): container finished" podID="33381711-7e71-4b61-bf6a-e101f47a717b" containerID="7258e6b3b21108400463368b892ee21247afab3ccc987acd79fe5320a85014ba" exitCode=0 Jul 18 02:06:26.336160 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:26.335761 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" event={"ID":"33381711-7e71-4b61-bf6a-e101f47a717b","Type":"ContainerDied","Data":"7258e6b3b21108400463368b892ee21247afab3ccc987acd79fe5320a85014ba"} Jul 18 02:06:26.336208 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:26.336162 2573 scope.go:117] "RemoveContainer" containerID="7258e6b3b21108400463368b892ee21247afab3ccc987acd79fe5320a85014ba" Jul 18 02:06:26.337072 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:26.337046 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:06:26.337187 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:26.337108 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:06:26.337187 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:26.337141 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:06:27.342414 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:27.342375 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" event={"ID":"33381711-7e71-4b61-bf6a-e101f47a717b","Type":"ContainerStarted","Data":"f296135a2008f92a6ea3428922fdc87171903a6cd27ffae5baaf4bf8575130ac"} Jul 18 02:06:27.843002 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:27.842934 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:06:27.844134 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:27.844104 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zt886 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 18 02:06:27.844290 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:27.844164 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" podUID="33381711-7e71-4b61-bf6a-e101f47a717b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 18 02:06:28.842379 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:28.842330 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zt886 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 18 02:06:28.842784 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:28.842393 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" podUID="33381711-7e71-4b61-bf6a-e101f47a717b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 18 02:06:29.715482 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:29.715444 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:29.804072 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:29.804041 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:29.842172 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:29.842133 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zt886 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 18 02:06:29.842346 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:29.842196 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" podUID="33381711-7e71-4b61-bf6a-e101f47a717b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 18 02:06:29.874934 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:29.874906 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:06:30.842173 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:30.842135 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zt886 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 18 02:06:30.842350 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:30.842192 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" podUID="33381711-7e71-4b61-bf6a-e101f47a717b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 18 02:06:31.846068 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:31.846039 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:06:31.846867 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:31.846847 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zt886" Jul 18 02:06:33.321583 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:33.321552 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-986hh" Jul 18 02:06:37.446986 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:37.446941 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:37.495965 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:37.495934 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:41.638913 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:41.638872 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:49.502877 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:49.502842 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:50.031989 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.031934 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l"] Jul 18 02:06:50.035816 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.035792 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.038399 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.038375 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 18 02:06:50.039279 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.039260 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7gdmh\"" Jul 18 02:06:50.039390 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.039343 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 18 02:06:50.039390 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.039370 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 18 02:06:50.045765 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.045731 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l"] Jul 18 02:06:50.066510 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.066468 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.066510 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.066506 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.066737 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.066679 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvs92\" (UniqueName: \"kubernetes.io/projected/b847ea02-1f8c-4b8e-b60d-6ca14611327d-kube-api-access-zvs92\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.066800 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.066777 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.066855 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.066819 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b847ea02-1f8c-4b8e-b60d-6ca14611327d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.066905 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.066866 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.071051 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.070998 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:06:50.167859 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.167818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvs92\" (UniqueName: \"kubernetes.io/projected/b847ea02-1f8c-4b8e-b60d-6ca14611327d-kube-api-access-zvs92\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.168081 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.167871 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.168081 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.168030 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b847ea02-1f8c-4b8e-b60d-6ca14611327d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.168190 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.168084 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.168190 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.168120 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.168190 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.168139 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.168343 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.168256 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.168477 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.168458 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.168552 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.168518 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.170268 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.170250 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b847ea02-1f8c-4b8e-b60d-6ca14611327d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.170593 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.170572 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b847ea02-1f8c-4b8e-b60d-6ca14611327d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.176021 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.175964 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvs92\" (UniqueName: \"kubernetes.io/projected/b847ea02-1f8c-4b8e-b60d-6ca14611327d-kube-api-access-zvs92\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wft9l\" (UID: \"b847ea02-1f8c-4b8e-b60d-6ca14611327d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.347376 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.347272 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:06:50.485049 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:50.485019 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l"] Jul 18 02:06:50.486779 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:06:50.486748 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb847ea02_1f8c_4b8e_b60d_6ca14611327d.slice/crio-214391625fba00acd4deaddb7281951b6400709f550a041db6ef3bc6cefd2418 WatchSource:0}: Error finding container 214391625fba00acd4deaddb7281951b6400709f550a041db6ef3bc6cefd2418: Status 404 returned error can't find the container with id 214391625fba00acd4deaddb7281951b6400709f550a041db6ef3bc6cefd2418 Jul 18 02:06:51.465751 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:51.465661 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" event={"ID":"b847ea02-1f8c-4b8e-b60d-6ca14611327d","Type":"ContainerStarted","Data":"214391625fba00acd4deaddb7281951b6400709f550a041db6ef3bc6cefd2418"} Jul 18 02:06:52.472119 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:52.472087 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" event={"ID":"b847ea02-1f8c-4b8e-b60d-6ca14611327d","Type":"ContainerStarted","Data":"df834e21a53a24495de03cbacae96bccd23eb79433fdb221c799d8eebe8443d6"} Jul 18 02:06:52.489291 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:52.489241 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" podStartSLOduration=1.407863958 podStartE2EDuration="2.489222111s" totalImagesPullingTime="1.081358153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:06:52.488597615 +0000 UTC m=+600.276934198" watchObservedRunningTime="2026-07-18 02:06:52.489222111 +0000 UTC m=+600.277558693" Jul 18 02:06:52.632307 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:52.632274 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:06:52.632559 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:52.632545 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:06:56.132532 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:06:56.132499 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:07:00.348248 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:00.348194 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:07:00.348668 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:00.348274 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:07:00.360717 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:00.360689 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:07:00.520403 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:00.520374 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wft9l" Jul 18 02:07:26.172355 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:26.172256 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:07:48.529863 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.529827 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk"] Jul 18 02:07:48.533083 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.533067 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.537752 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.537730 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 18 02:07:48.550492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.550464 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk"] Jul 18 02:07:48.560669 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.560645 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.560802 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.560678 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.560802 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.560702 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.560802 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.560724 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w54xp\" (UniqueName: \"kubernetes.io/projected/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-kube-api-access-w54xp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.560906 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.560801 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.560906 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.560858 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.597509 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.597477 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:07:48.662350 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.662305 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.662350 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.662356 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.662620 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.662384 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.662620 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.662411 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w54xp\" (UniqueName: \"kubernetes.io/projected/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-kube-api-access-w54xp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.662620 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.662442 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.662620 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.662478 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.662830 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.662753 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.662830 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.662783 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.662937 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.662869 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.664762 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.664739 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.665053 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.665035 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.684151 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.684119 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w54xp\" (UniqueName: \"kubernetes.io/projected/2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8-kube-api-access-w54xp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hzzk\" (UID: \"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.842719 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.842621 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:48.977200 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.977127 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk"] Jul 18 02:07:48.979544 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:07:48.979518 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc2d127_c82e_41e8_aff0_9d5a5fd3b3e8.slice/crio-94a4e00ad2d7c1150e5e7671724a8b32d7ea84f255aac9c242fd846d40cb4569 WatchSource:0}: Error finding container 94a4e00ad2d7c1150e5e7671724a8b32d7ea84f255aac9c242fd846d40cb4569: Status 404 returned error can't find the container with id 94a4e00ad2d7c1150e5e7671724a8b32d7ea84f255aac9c242fd846d40cb4569 Jul 18 02:07:48.980797 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:48.980776 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:07:49.765370 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:49.765336 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" event={"ID":"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8","Type":"ContainerStarted","Data":"a92fe808126fb871d44c2cdc7a99a6bcb02b40a2827933754b2c40711d4cbeb4"} Jul 18 02:07:49.765786 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:49.765389 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" event={"ID":"2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8","Type":"ContainerStarted","Data":"94a4e00ad2d7c1150e5e7671724a8b32d7ea84f255aac9c242fd846d40cb4569"} Jul 18 02:07:49.780658 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:49.780603 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" podStartSLOduration=1.595325629 podStartE2EDuration="1.780586114s" totalImagesPullingTime="185.260485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:07:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:07:49.779843322 +0000 UTC m=+657.568179904" watchObservedRunningTime="2026-07-18 02:07:49.780586114 +0000 UTC m=+657.568922769" Jul 18 02:07:52.364015 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:52.363965 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:07:58.843120 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:58.843075 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:58.843665 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:58.843148 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:58.855688 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:58.855657 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:07:59.826895 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:07:59.826861 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hzzk" Jul 18 02:09:23.094264 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.094175 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-568fcc8d45-5r86j"] Jul 18 02:09:23.097329 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.097311 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-568fcc8d45-5r86j" Jul 18 02:09:23.106803 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.106777 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-568fcc8d45-5r86j"] Jul 18 02:09:23.274200 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.274157 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4llhc\" (UniqueName: \"kubernetes.io/projected/b0269a7b-591d-4c64-889c-1dcb1660eadf-kube-api-access-4llhc\") pod \"authorino-568fcc8d45-5r86j\" (UID: \"b0269a7b-591d-4c64-889c-1dcb1660eadf\") " pod="kuadrant-system/authorino-568fcc8d45-5r86j" Jul 18 02:09:23.274375 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.274262 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0269a7b-591d-4c64-889c-1dcb1660eadf-tls-cert\") pod \"authorino-568fcc8d45-5r86j\" (UID: \"b0269a7b-591d-4c64-889c-1dcb1660eadf\") " pod="kuadrant-system/authorino-568fcc8d45-5r86j" Jul 18 02:09:23.375151 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.375125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0269a7b-591d-4c64-889c-1dcb1660eadf-tls-cert\") pod \"authorino-568fcc8d45-5r86j\" (UID: \"b0269a7b-591d-4c64-889c-1dcb1660eadf\") " pod="kuadrant-system/authorino-568fcc8d45-5r86j" Jul 18 02:09:23.375318 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.375196 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4llhc\" (UniqueName: \"kubernetes.io/projected/b0269a7b-591d-4c64-889c-1dcb1660eadf-kube-api-access-4llhc\") pod \"authorino-568fcc8d45-5r86j\" (UID: \"b0269a7b-591d-4c64-889c-1dcb1660eadf\") " pod="kuadrant-system/authorino-568fcc8d45-5r86j" Jul 18 02:09:23.377442 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.377420 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0269a7b-591d-4c64-889c-1dcb1660eadf-tls-cert\") pod \"authorino-568fcc8d45-5r86j\" (UID: \"b0269a7b-591d-4c64-889c-1dcb1660eadf\") " pod="kuadrant-system/authorino-568fcc8d45-5r86j" Jul 18 02:09:23.382204 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.382180 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4llhc\" (UniqueName: \"kubernetes.io/projected/b0269a7b-591d-4c64-889c-1dcb1660eadf-kube-api-access-4llhc\") pod \"authorino-568fcc8d45-5r86j\" (UID: \"b0269a7b-591d-4c64-889c-1dcb1660eadf\") " pod="kuadrant-system/authorino-568fcc8d45-5r86j" Jul 18 02:09:23.406589 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.406561 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-568fcc8d45-5r86j" Jul 18 02:09:23.537053 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:23.537023 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-568fcc8d45-5r86j"] Jul 18 02:09:23.538934 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:09:23.538903 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0269a7b_591d_4c64_889c_1dcb1660eadf.slice/crio-39f1c56eea374559b46650415d637c402fc1c36e89a4e2e82ebb87e83fd27281 WatchSource:0}: Error finding container 39f1c56eea374559b46650415d637c402fc1c36e89a4e2e82ebb87e83fd27281: Status 404 returned error can't find the container with id 39f1c56eea374559b46650415d637c402fc1c36e89a4e2e82ebb87e83fd27281 Jul 18 02:09:24.223158 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.223053 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-568fcc8d45-5r86j" event={"ID":"b0269a7b-591d-4c64-889c-1dcb1660eadf","Type":"ContainerStarted","Data":"44908ca88bc257655561294fca2417c78aca7771353c68c1a7d33886d3fcb7f9"} Jul 18 02:09:24.223158 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.223105 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-568fcc8d45-5r86j" event={"ID":"b0269a7b-591d-4c64-889c-1dcb1660eadf","Type":"ContainerStarted","Data":"39f1c56eea374559b46650415d637c402fc1c36e89a4e2e82ebb87e83fd27281"} Jul 18 02:09:24.244703 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.244641 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-568fcc8d45-5r86j" podStartSLOduration=0.836722247 podStartE2EDuration="1.244625228s" totalImagesPullingTime="407.902981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:09:24.24273045 +0000 UTC m=+752.031067032" watchObservedRunningTime="2026-07-18 02:09:24.244625228 +0000 UTC m=+752.032961808" Jul 18 02:09:24.272555 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.272512 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6859878464-pcp6z"] Jul 18 02:09:24.272757 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.272711 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6859878464-pcp6z" podUID="0817f2bc-39a2-47cf-8ff2-2d2f194ba30c" containerName="authorino" containerID="cri-o://c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b" gracePeriod=30 Jul 18 02:09:24.550900 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.550872 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6859878464-pcp6z" Jul 18 02:09:24.685827 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.685791 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-tls-cert\") pod \"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c\" (UID: \"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c\") " Jul 18 02:09:24.686036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.685901 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m42tt\" (UniqueName: \"kubernetes.io/projected/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-kube-api-access-m42tt\") pod \"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c\" (UID: \"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c\") " Jul 18 02:09:24.688010 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.687952 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-kube-api-access-m42tt" (OuterVolumeSpecName: "kube-api-access-m42tt") pod "0817f2bc-39a2-47cf-8ff2-2d2f194ba30c" (UID: "0817f2bc-39a2-47cf-8ff2-2d2f194ba30c"). InnerVolumeSpecName "kube-api-access-m42tt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:09:24.696307 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.696271 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0817f2bc-39a2-47cf-8ff2-2d2f194ba30c" (UID: "0817f2bc-39a2-47cf-8ff2-2d2f194ba30c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:09:24.786698 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.786597 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m42tt\" (UniqueName: \"kubernetes.io/projected/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-kube-api-access-m42tt\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:09:24.786698 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.786641 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c-tls-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:09:24.815042 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.814998 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:09:24.901351 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:24.901312 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:09:25.230711 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:25.230672 2573 generic.go:356] "Generic (PLEG): container finished" podID="0817f2bc-39a2-47cf-8ff2-2d2f194ba30c" containerID="c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b" exitCode=0 Jul 18 02:09:25.231214 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:25.230716 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6859878464-pcp6z" Jul 18 02:09:25.231214 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:25.230757 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6859878464-pcp6z" event={"ID":"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c","Type":"ContainerDied","Data":"c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b"} Jul 18 02:09:25.231214 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:25.230802 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6859878464-pcp6z" event={"ID":"0817f2bc-39a2-47cf-8ff2-2d2f194ba30c","Type":"ContainerDied","Data":"2792d77149f904989039039abeb0e7bca942d70ba50e3eabbc44e3996ef9ad5a"} Jul 18 02:09:25.231214 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:25.230824 2573 scope.go:117] "RemoveContainer" containerID="c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b" Jul 18 02:09:25.239627 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:25.239604 2573 scope.go:117] "RemoveContainer" containerID="c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b" Jul 18 02:09:25.239919 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:09:25.239897 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b\": container with ID starting with c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b not found: ID does not exist" containerID="c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b" Jul 18 02:09:25.240018 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:25.239933 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b"} err="failed to get container status \"c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b\": rpc error: code = NotFound desc = could not find container \"c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b\": container with ID starting with c6da2fbc2907ca0561a0af0305315ecf0e9ad8fdd57902d78b8c1691e9e6231b not found: ID does not exist" Jul 18 02:09:25.246336 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:25.246305 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6859878464-pcp6z"] Jul 18 02:09:25.249946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:25.249922 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6859878464-pcp6z"] Jul 18 02:09:25.910775 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:25.910743 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:09:26.018138 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:26.018111 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:09:26.722944 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:09:26.722911 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0817f2bc-39a2-47cf-8ff2-2d2f194ba30c" path="/var/lib/kubelet/pods/0817f2bc-39a2-47cf-8ff2-2d2f194ba30c/volumes" Jul 18 02:10:19.096550 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:10:19.096451 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:10:19.198541 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:10:19.198500 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:10:36.871513 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:10:36.871481 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:11:29.693626 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:11:29.693592 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:11:30.933844 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:11:30.933806 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:11:32.832519 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:11:32.832486 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:11:52.663598 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:11:52.663520 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:11:52.664304 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:11:52.664285 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:12:06.366283 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:12:06.366241 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:12:07.359575 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:12:07.359542 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:12:12.227374 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:12:12.227332 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:12:12.332458 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:12:12.332421 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:12:12.432260 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:12:12.432224 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:12:12.574923 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:12:12.574840 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:12:18.104561 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:12:18.104525 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:12:18.205386 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:12:18.205353 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:12:50.683431 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:12:50.683390 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:13:16.431642 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:13:16.431557 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:13:30.028172 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:13:30.028136 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:16:52.696030 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:16:52.696001 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:16:52.697193 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:16:52.697171 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:18:55.653458 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:18:55.653427 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:20:08.083390 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:20:08.083360 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:20:08.137316 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:20:08.137279 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:21:52.729376 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:21:52.729345 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:21:52.732755 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:21:52.732737 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:22:54.297235 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:22:54.297197 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:03.464080 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:03.464042 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:12.972184 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:12.972150 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:21.236603 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:21.236568 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:26.315114 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:26.315075 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:29.219163 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:29.219121 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:29.321745 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:29.321712 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:29.695785 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:29.695745 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:35.704600 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:35.704557 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:35.806303 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:35.806272 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:35.928906 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:35.928877 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:36.229919 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.229886 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs"] Jul 18 02:23:36.230216 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.230204 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0817f2bc-39a2-47cf-8ff2-2d2f194ba30c" containerName="authorino" Jul 18 02:23:36.230261 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.230217 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0817f2bc-39a2-47cf-8ff2-2d2f194ba30c" containerName="authorino" Jul 18 02:23:36.230335 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.230326 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0817f2bc-39a2-47cf-8ff2-2d2f194ba30c" containerName="authorino" Jul 18 02:23:36.233197 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.233180 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.236054 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.236034 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-914a7c26-openshift-default-dockercfg-4hpzm\"" Jul 18 02:23:36.251580 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.251558 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:36.268498 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.268477 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs"] Jul 18 02:23:36.383765 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.383734 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n92m\" (UniqueName: \"kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-kube-api-access-8n92m\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.383940 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.383772 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946948b4-5c6d-44a5-8074-95d0fec4e419-istiod-ca-cert\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.383940 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.383797 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-envoy\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.383940 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.383862 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-token\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.383940 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.383885 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-credential-socket\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.383940 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.383912 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-certs\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.384160 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.383950 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-data\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.384160 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.383995 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-socket\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.384160 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.384010 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-podinfo\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.405757 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.405723 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:36.485077 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.484997 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-token\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485077 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485035 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-credential-socket\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485283 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485163 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-certs\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485283 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485217 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-data\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485283 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485237 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-socket\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485283 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485260 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-podinfo\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485468 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485291 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n92m\" (UniqueName: \"kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-kube-api-access-8n92m\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485468 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485315 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946948b4-5c6d-44a5-8074-95d0fec4e419-istiod-ca-cert\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485468 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-envoy\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485468 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485443 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-credential-socket\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485672 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485529 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-certs\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485768 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485748 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-socket\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.485865 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.485834 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-data\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.486110 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.486088 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946948b4-5c6d-44a5-8074-95d0fec4e419-istiod-ca-cert\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.487512 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.487483 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-envoy\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.487642 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.487625 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-podinfo\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.495127 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.495103 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n92m\" (UniqueName: \"kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-kube-api-access-8n92m\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.495453 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.495433 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-token\") pod \"e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.542942 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.542915 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:36.674801 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.674776 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs"] Jul 18 02:23:36.676951 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:23:36.676916 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946948b4_5c6d_44a5_8074_95d0fec4e419.slice/crio-e3a887af9718aaa4d4b544a089c9c4933abf417079ccb0a8a50a386705ab491b WatchSource:0}: Error finding container e3a887af9718aaa4d4b544a089c9c4933abf417079ccb0a8a50a386705ab491b: Status 404 returned error can't find the container with id e3a887af9718aaa4d4b544a089c9c4933abf417079ccb0a8a50a386705ab491b Jul 18 02:23:36.679005 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.678948 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:23:36.679078 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.679033 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:23:36.679078 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:36.679067 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:23:37.355650 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:37.355616 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" event={"ID":"946948b4-5c6d-44a5-8074-95d0fec4e419","Type":"ContainerStarted","Data":"a96d910d6a550ee1d3c9a7e77e1cf1d14da983793acd111032ee23c145d0e516"} Jul 18 02:23:37.356055 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:37.355674 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" event={"ID":"946948b4-5c6d-44a5-8074-95d0fec4e419","Type":"ContainerStarted","Data":"e3a887af9718aaa4d4b544a089c9c4933abf417079ccb0a8a50a386705ab491b"} Jul 18 02:23:37.378890 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:37.378846 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" podStartSLOduration=1.378833484 podStartE2EDuration="1.378833484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:23:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:23:37.376739508 +0000 UTC m=+1605.165076089" watchObservedRunningTime="2026-07-18 02:23:37.378833484 +0000 UTC m=+1605.167170066" Jul 18 02:23:37.543267 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:37.543233 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:37.547882 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:37.547860 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:38.360014 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:38.359959 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:38.360982 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:38.360952 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:38.732013 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:38.731958 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:38.821163 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:38.821131 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:38.969098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:38.969064 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:39.116730 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:39.116637 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:39.475721 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:39.475690 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:39.578083 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:39.578055 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:23:52.433949 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:52.433915 2573 generic.go:356] "Generic (PLEG): container finished" podID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerID="a96d910d6a550ee1d3c9a7e77e1cf1d14da983793acd111032ee23c145d0e516" exitCode=0 Jul 18 02:23:52.433949 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:52.433951 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" event={"ID":"946948b4-5c6d-44a5-8074-95d0fec4e419","Type":"ContainerDied","Data":"a96d910d6a550ee1d3c9a7e77e1cf1d14da983793acd111032ee23c145d0e516"} Jul 18 02:23:52.434488 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:52.434344 2573 scope.go:117] "RemoveContainer" containerID="a96d910d6a550ee1d3c9a7e77e1cf1d14da983793acd111032ee23c145d0e516" Jul 18 02:23:52.435326 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:52.435292 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:23:52.435431 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:52.435364 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:23:52.435431 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:52.435394 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:23:53.360206 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:53.360166 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:53.440219 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:53.440181 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" event={"ID":"946948b4-5c6d-44a5-8074-95d0fec4e419","Type":"ContainerStarted","Data":"966c3c2971480570bff06db3131e8b553207f1d02fedeecac005880ac97ee60a"} Jul 18 02:23:53.543396 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:53.543356 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:53.544374 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:53.544349 2573 patch_prober.go:28] interesting pod/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 18 02:23:53.544502 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:53.544397 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 18 02:23:54.543778 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:54.543744 2573 patch_prober.go:28] interesting pod/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 18 02:23:54.544231 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:54.543809 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 18 02:23:55.544315 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:55.544277 2573 patch_prober.go:28] interesting pod/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 18 02:23:55.544802 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:55.544367 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 18 02:23:56.543875 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:56.543835 2573 patch_prober.go:28] interesting pod/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 18 02:23:56.544123 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:56.543895 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 18 02:23:57.547495 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:57.547469 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:57.547945 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:57.547805 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:23:57.548488 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:23:57.548472 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:24:14.750716 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:14.750670 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:19.332372 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:19.332336 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:19.472791 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:19.472759 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs"] Jul 18 02:24:19.473041 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:19.473014 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" containerID="cri-o://966c3c2971480570bff06db3131e8b553207f1d02fedeecac005880ac97ee60a" gracePeriod=30 Jul 18 02:24:19.497233 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:19.497200 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:19.740769 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:19.740738 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:19.987530 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:19.987493 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:20.089770 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.089681 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:20.187031 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.187003 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:20.483557 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.483525 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d"] Jul 18 02:24:20.487126 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.487104 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.491326 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.491306 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-5e522e36-gw-openshift-default-dockercfg-xg2bl\"" Jul 18 02:24:20.502035 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.502013 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d"] Jul 18 02:24:20.525194 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.525158 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:20.612302 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.612267 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-certs\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.612302 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.612310 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv2rk\" (UniqueName: \"kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-kube-api-access-xv2rk\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.612596 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.612338 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f59db8a-d840-431f-af58-fffca9055406-istiod-ca-cert\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.612596 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.612388 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-istio-token\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.612596 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.612439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f59db8a-d840-431f-af58-fffca9055406-istio-podinfo\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.612596 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.612473 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-credential-socket\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.612596 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.612557 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-socket\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.612596 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.612587 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-envoy\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.612842 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.612608 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-data\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.713554 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.713513 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-socket\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.713554 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.713567 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-envoy\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.713824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.713590 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-data\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.713824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.713668 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-certs\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.713824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.713697 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv2rk\" (UniqueName: \"kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-kube-api-access-xv2rk\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.713824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.713735 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f59db8a-d840-431f-af58-fffca9055406-istiod-ca-cert\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.713824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.713757 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-istio-token\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.713824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.713791 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f59db8a-d840-431f-af58-fffca9055406-istio-podinfo\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.713824 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.713825 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-credential-socket\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.714191 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.713903 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-socket\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.714322 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.714300 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-credential-socket\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.714394 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.714328 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-certs\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.714509 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.714495 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-data\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.714562 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.714539 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f59db8a-d840-431f-af58-fffca9055406-istiod-ca-cert\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.715888 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.715866 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-envoy\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.716705 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.716682 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f59db8a-d840-431f-af58-fffca9055406-istio-podinfo\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.729349 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.729321 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv2rk\" (UniqueName: \"kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-kube-api-access-xv2rk\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.731453 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.731430 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-istio-token\") pod \"e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.799839 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.799747 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:20.837265 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.837232 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:20.937030 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.937005 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d"] Jul 18 02:24:20.939333 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:24:20.939306 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f59db8a_d840_431f_af58_fffca9055406.slice/crio-e2795933b896004237f0898ee0c617bbcf08079a1f8d3380bc551b3b4550a173 WatchSource:0}: Error finding container e2795933b896004237f0898ee0c617bbcf08079a1f8d3380bc551b3b4550a173: Status 404 returned error can't find the container with id e2795933b896004237f0898ee0c617bbcf08079a1f8d3380bc551b3b4550a173 Jul 18 02:24:20.941478 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.941444 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:24:20.941564 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.941510 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:24:20.941564 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:20.941538 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:24:21.578111 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:21.578074 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" event={"ID":"2f59db8a-d840-431f-af58-fffca9055406","Type":"ContainerStarted","Data":"479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260"} Jul 18 02:24:21.578489 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:21.578126 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" event={"ID":"2f59db8a-d840-431f-af58-fffca9055406","Type":"ContainerStarted","Data":"e2795933b896004237f0898ee0c617bbcf08079a1f8d3380bc551b3b4550a173"} Jul 18 02:24:21.598865 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:21.598815 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" podStartSLOduration=1.598801074 podStartE2EDuration="1.598801074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:24:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:24:21.597107493 +0000 UTC m=+1649.385444074" watchObservedRunningTime="2026-07-18 02:24:21.598801074 +0000 UTC m=+1649.387137655" Jul 18 02:24:21.800761 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:21.800725 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:21.806043 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:21.806017 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:22.583283 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:22.583257 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:22.584129 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:22.584117 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:23.076897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:23.076855 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:23.281444 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:23.281415 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:23.851746 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:23.851702 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:23.955693 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:23.955655 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:24.052165 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.052133 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:24:24.595100 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.595072 2573 generic.go:356] "Generic (PLEG): container finished" podID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerID="966c3c2971480570bff06db3131e8b553207f1d02fedeecac005880ac97ee60a" exitCode=0 Jul 18 02:24:24.595247 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.595105 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" event={"ID":"946948b4-5c6d-44a5-8074-95d0fec4e419","Type":"ContainerDied","Data":"966c3c2971480570bff06db3131e8b553207f1d02fedeecac005880ac97ee60a"} Jul 18 02:24:24.595247 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.595178 2573 scope.go:117] "RemoveContainer" containerID="a96d910d6a550ee1d3c9a7e77e1cf1d14da983793acd111032ee23c145d0e516" Jul 18 02:24:24.726655 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.726626 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:24:24.748078 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.747989 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n92m\" (UniqueName: \"kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-kube-api-access-8n92m\") pod \"946948b4-5c6d-44a5-8074-95d0fec4e419\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " Jul 18 02:24:24.748078 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748036 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-podinfo\") pod \"946948b4-5c6d-44a5-8074-95d0fec4e419\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " Jul 18 02:24:24.748078 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748059 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-credential-socket\") pod \"946948b4-5c6d-44a5-8074-95d0fec4e419\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " Jul 18 02:24:24.748078 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748082 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-socket\") pod \"946948b4-5c6d-44a5-8074-95d0fec4e419\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " Jul 18 02:24:24.748403 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748128 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-certs\") pod \"946948b4-5c6d-44a5-8074-95d0fec4e419\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " Jul 18 02:24:24.748403 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748147 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-token\") pod \"946948b4-5c6d-44a5-8074-95d0fec4e419\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " Jul 18 02:24:24.748403 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748181 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-data\") pod \"946948b4-5c6d-44a5-8074-95d0fec4e419\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " Jul 18 02:24:24.748403 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748207 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-envoy\") pod \"946948b4-5c6d-44a5-8074-95d0fec4e419\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " Jul 18 02:24:24.748403 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748249 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946948b4-5c6d-44a5-8074-95d0fec4e419-istiod-ca-cert\") pod \"946948b4-5c6d-44a5-8074-95d0fec4e419\" (UID: \"946948b4-5c6d-44a5-8074-95d0fec4e419\") " Jul 18 02:24:24.748655 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748416 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "946948b4-5c6d-44a5-8074-95d0fec4e419" (UID: "946948b4-5c6d-44a5-8074-95d0fec4e419"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:24:24.748736 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748703 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/946948b4-5c6d-44a5-8074-95d0fec4e419-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "946948b4-5c6d-44a5-8074-95d0fec4e419" (UID: "946948b4-5c6d-44a5-8074-95d0fec4e419"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:24:24.748857 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748833 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "946948b4-5c6d-44a5-8074-95d0fec4e419" (UID: "946948b4-5c6d-44a5-8074-95d0fec4e419"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:24:24.749053 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.748853 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "946948b4-5c6d-44a5-8074-95d0fec4e419" (UID: "946948b4-5c6d-44a5-8074-95d0fec4e419"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:24:24.749350 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.749330 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-data" (OuterVolumeSpecName: "istio-data") pod "946948b4-5c6d-44a5-8074-95d0fec4e419" (UID: "946948b4-5c6d-44a5-8074-95d0fec4e419"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:24:24.750497 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.750445 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-kube-api-access-8n92m" (OuterVolumeSpecName: "kube-api-access-8n92m") pod "946948b4-5c6d-44a5-8074-95d0fec4e419" (UID: "946948b4-5c6d-44a5-8074-95d0fec4e419"). InnerVolumeSpecName "kube-api-access-8n92m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:24:24.750914 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.750896 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "946948b4-5c6d-44a5-8074-95d0fec4e419" (UID: "946948b4-5c6d-44a5-8074-95d0fec4e419"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:24:24.751106 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.751076 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "946948b4-5c6d-44a5-8074-95d0fec4e419" (UID: "946948b4-5c6d-44a5-8074-95d0fec4e419"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:24:24.751187 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.751111 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-token" (OuterVolumeSpecName: "istio-token") pod "946948b4-5c6d-44a5-8074-95d0fec4e419" (UID: "946948b4-5c6d-44a5-8074-95d0fec4e419"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:24:24.849167 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.849130 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:24:24.849167 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.849160 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-token\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:24:24.849167 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.849167 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-data\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:24:24.849167 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.849176 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-envoy\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:24:24.849431 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.849185 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946948b4-5c6d-44a5-8074-95d0fec4e419-istiod-ca-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:24:24.849431 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.849194 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n92m\" (UniqueName: \"kubernetes.io/projected/946948b4-5c6d-44a5-8074-95d0fec4e419-kube-api-access-8n92m\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:24:24.849431 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.849202 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946948b4-5c6d-44a5-8074-95d0fec4e419-istio-podinfo\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:24:24.849431 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.849210 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-credential-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:24:24.849431 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:24.849217 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946948b4-5c6d-44a5-8074-95d0fec4e419-workload-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:24:25.600754 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:25.600720 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" Jul 18 02:24:25.601234 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:25.600777 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs" event={"ID":"946948b4-5c6d-44a5-8074-95d0fec4e419","Type":"ContainerDied","Data":"e3a887af9718aaa4d4b544a089c9c4933abf417079ccb0a8a50a386705ab491b"} Jul 18 02:24:25.601234 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:25.600832 2573 scope.go:117] "RemoveContainer" containerID="966c3c2971480570bff06db3131e8b553207f1d02fedeecac005880ac97ee60a" Jul 18 02:24:25.621412 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:25.621378 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs"] Jul 18 02:24:25.626868 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:25.626837 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-914a7c26-openshift-default-bd58df5cc-85xrs"] Jul 18 02:24:26.722515 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:26.722480 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" path="/var/lib/kubelet/pods/946948b4-5c6d-44a5-8074-95d0fec4e419/volumes" Jul 18 02:24:36.662009 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:36.661959 2573 generic.go:356] "Generic (PLEG): container finished" podID="2f59db8a-d840-431f-af58-fffca9055406" containerID="479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260" exitCode=0 Jul 18 02:24:36.662445 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:36.662029 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" event={"ID":"2f59db8a-d840-431f-af58-fffca9055406","Type":"ContainerDied","Data":"479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260"} Jul 18 02:24:36.662445 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:36.662404 2573 scope.go:117] "RemoveContainer" containerID="479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260" Jul 18 02:24:36.663327 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:36.663297 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:24:36.663430 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:36.663375 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:24:36.663430 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:36.663423 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:24:37.584218 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:37.584163 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:37.668611 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:37.668579 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" event={"ID":"2f59db8a-d840-431f-af58-fffca9055406","Type":"ContainerStarted","Data":"c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f"} Jul 18 02:24:37.800690 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:37.800653 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:37.801858 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:37.801823 2573 patch_prober.go:28] interesting pod/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 18 02:24:37.802011 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:37.801885 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 18 02:24:38.800437 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:38.800401 2573 patch_prober.go:28] interesting pod/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 18 02:24:38.801039 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:38.800995 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 18 02:24:39.800941 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:39.800891 2573 patch_prober.go:28] interesting pod/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 18 02:24:39.801389 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:39.800961 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 18 02:24:40.802454 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:40.800909 2573 patch_prober.go:28] interesting pod/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 18 02:24:40.802454 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:40.801010 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 18 02:24:41.804421 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:41.804393 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:41.804796 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:41.804655 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:41.805342 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:41.805322 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:24:58.771528 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:24:58.771491 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:03.201065 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:03.201029 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:03.370548 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:03.370515 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d"] Jul 18 02:25:03.370744 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:03.370724 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" containerID="cri-o://c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f" gracePeriod=30 Jul 18 02:25:03.395662 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:03.395632 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:03.630648 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:03.630610 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:03.926817 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:03.926736 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:04.238999 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.238912 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c"] Jul 18 02:25:04.239360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.239284 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" Jul 18 02:25:04.239360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.239295 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" Jul 18 02:25:04.239360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.239306 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" Jul 18 02:25:04.239360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.239312 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" Jul 18 02:25:04.239499 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.239379 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" Jul 18 02:25:04.242321 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.242305 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.245328 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.245306 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-eaf8d750-openshift-default-dockercfg-vww6v\"" Jul 18 02:25:04.260031 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.259956 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c"] Jul 18 02:25:04.273661 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.273618 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:04.406353 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.406310 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c63903d-7726-4151-a3c1-2734510dc275-istiod-ca-cert\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.406353 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.406361 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zzzf\" (UniqueName: \"kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-kube-api-access-5zzzf\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.406593 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.406400 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-istio-token\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.406593 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.406439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-credential-socket\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.406593 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.406504 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-envoy\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.406593 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.406576 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c63903d-7726-4151-a3c1-2734510dc275-istio-podinfo\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.406745 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.406597 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-certs\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.406745 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.406611 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-data\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.406745 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.406675 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-socket\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.507822 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.507716 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-socket\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.507822 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.507761 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c63903d-7726-4151-a3c1-2734510dc275-istiod-ca-cert\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.507822 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.507782 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zzzf\" (UniqueName: \"kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-kube-api-access-5zzzf\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508162 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.507897 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-istio-token\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508162 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.507956 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-credential-socket\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508162 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.508026 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-envoy\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508290 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.508161 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-socket\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508290 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.508279 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-credential-socket\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508386 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.508372 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c63903d-7726-4151-a3c1-2734510dc275-istio-podinfo\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508421 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.508396 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-certs\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508465 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.508428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-data\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508562 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.508539 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c63903d-7726-4151-a3c1-2734510dc275-istiod-ca-cert\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508665 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.508640 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-certs\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.508817 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.508750 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-data\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.510654 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.510632 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c63903d-7726-4151-a3c1-2734510dc275-istio-podinfo\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.510759 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.510677 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-envoy\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.519480 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.519458 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zzzf\" (UniqueName: \"kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-kube-api-access-5zzzf\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.519480 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.519468 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-istio-token\") pod \"e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.553379 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.553343 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:04.577061 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.577026 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:04.693718 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.693693 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c"] Jul 18 02:25:04.695245 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:25:04.695218 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c63903d_7726_4151_a3c1_2734510dc275.slice/crio-c6264f03c1733a7930d9dd5b231d20147e66b697143ed3bbc6bb39107b5ef6a5 WatchSource:0}: Error finding container c6264f03c1733a7930d9dd5b231d20147e66b697143ed3bbc6bb39107b5ef6a5: Status 404 returned error can't find the container with id c6264f03c1733a7930d9dd5b231d20147e66b697143ed3bbc6bb39107b5ef6a5 Jul 18 02:25:04.697308 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.697278 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:25:04.697381 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.697343 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:25:04.697381 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.697369 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:25:04.810003 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.809875 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" event={"ID":"9c63903d-7726-4151-a3c1-2734510dc275","Type":"ContainerStarted","Data":"3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605"} Jul 18 02:25:04.810003 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.809940 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" event={"ID":"9c63903d-7726-4151-a3c1-2734510dc275","Type":"ContainerStarted","Data":"c6264f03c1733a7930d9dd5b231d20147e66b697143ed3bbc6bb39107b5ef6a5"} Jul 18 02:25:04.834032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:04.833962 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" podStartSLOduration=0.833945737 podStartE2EDuration="833.945737ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:25:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:25:04.831299105 +0000 UTC m=+1692.619635708" watchObservedRunningTime="2026-07-18 02:25:04.833945737 +0000 UTC m=+1692.622282318" Jul 18 02:25:05.553747 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:05.553709 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:05.558631 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:05.558608 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:05.815457 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:05.815373 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:05.816346 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:05.816326 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:06.274253 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:06.274222 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:06.371744 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:06.371711 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:06.446754 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:06.446694 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:06.640615 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:06.640583 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:06.950726 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:06.950635 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:07.051676 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:07.051643 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:08.631530 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.631503 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:25:08.749040 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.748927 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-istio-token\") pod \"2f59db8a-d840-431f-af58-fffca9055406\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " Jul 18 02:25:08.749040 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.748965 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-data\") pod \"2f59db8a-d840-431f-af58-fffca9055406\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " Jul 18 02:25:08.749040 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749033 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-socket\") pod \"2f59db8a-d840-431f-af58-fffca9055406\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " Jul 18 02:25:08.749327 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749072 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-certs\") pod \"2f59db8a-d840-431f-af58-fffca9055406\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " Jul 18 02:25:08.749327 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749093 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f59db8a-d840-431f-af58-fffca9055406-istio-podinfo\") pod \"2f59db8a-d840-431f-af58-fffca9055406\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " Jul 18 02:25:08.749327 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749137 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-envoy\") pod \"2f59db8a-d840-431f-af58-fffca9055406\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " Jul 18 02:25:08.749327 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749171 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-credential-socket\") pod \"2f59db8a-d840-431f-af58-fffca9055406\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " Jul 18 02:25:08.749327 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749217 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv2rk\" (UniqueName: \"kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-kube-api-access-xv2rk\") pod \"2f59db8a-d840-431f-af58-fffca9055406\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " Jul 18 02:25:08.749327 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749258 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f59db8a-d840-431f-af58-fffca9055406-istiod-ca-cert\") pod \"2f59db8a-d840-431f-af58-fffca9055406\" (UID: \"2f59db8a-d840-431f-af58-fffca9055406\") " Jul 18 02:25:08.749610 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749387 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2f59db8a-d840-431f-af58-fffca9055406" (UID: "2f59db8a-d840-431f-af58-fffca9055406"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:08.749610 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749382 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2f59db8a-d840-431f-af58-fffca9055406" (UID: "2f59db8a-d840-431f-af58-fffca9055406"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:08.749610 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749536 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:08.749610 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749561 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-workload-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:08.749610 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749526 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2f59db8a-d840-431f-af58-fffca9055406" (UID: "2f59db8a-d840-431f-af58-fffca9055406"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:08.749859 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749700 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f59db8a-d840-431f-af58-fffca9055406-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2f59db8a-d840-431f-af58-fffca9055406" (UID: "2f59db8a-d840-431f-af58-fffca9055406"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:25:08.750001 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.749957 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-data" (OuterVolumeSpecName: "istio-data") pod "2f59db8a-d840-431f-af58-fffca9055406" (UID: "2f59db8a-d840-431f-af58-fffca9055406"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:08.751230 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.751208 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-istio-token" (OuterVolumeSpecName: "istio-token") pod "2f59db8a-d840-431f-af58-fffca9055406" (UID: "2f59db8a-d840-431f-af58-fffca9055406"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:25:08.751329 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.751291 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f59db8a-d840-431f-af58-fffca9055406-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2f59db8a-d840-431f-af58-fffca9055406" (UID: "2f59db8a-d840-431f-af58-fffca9055406"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:25:08.751727 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.751698 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-kube-api-access-xv2rk" (OuterVolumeSpecName: "kube-api-access-xv2rk") pod "2f59db8a-d840-431f-af58-fffca9055406" (UID: "2f59db8a-d840-431f-af58-fffca9055406"). InnerVolumeSpecName "kube-api-access-xv2rk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:25:08.751727 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.751704 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2f59db8a-d840-431f-af58-fffca9055406" (UID: "2f59db8a-d840-431f-af58-fffca9055406"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:08.831756 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.831719 2573 generic.go:356] "Generic (PLEG): container finished" podID="2f59db8a-d840-431f-af58-fffca9055406" containerID="c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f" exitCode=0 Jul 18 02:25:08.831928 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.831766 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" Jul 18 02:25:08.831928 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.831803 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" event={"ID":"2f59db8a-d840-431f-af58-fffca9055406","Type":"ContainerDied","Data":"c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f"} Jul 18 02:25:08.831928 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.831856 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d" event={"ID":"2f59db8a-d840-431f-af58-fffca9055406","Type":"ContainerDied","Data":"e2795933b896004237f0898ee0c617bbcf08079a1f8d3380bc551b3b4550a173"} Jul 18 02:25:08.831928 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.831877 2573 scope.go:117] "RemoveContainer" containerID="c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f" Jul 18 02:25:08.841749 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.841726 2573 scope.go:117] "RemoveContainer" containerID="479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260" Jul 18 02:25:08.849836 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.849815 2573 scope.go:117] "RemoveContainer" containerID="c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f" Jul 18 02:25:08.850169 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:25:08.850149 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f\": container with ID starting with c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f not found: ID does not exist" containerID="c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f" Jul 18 02:25:08.850242 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.850177 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f"} err="failed to get container status \"c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f\": rpc error: code = NotFound desc = could not find container \"c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f\": container with ID starting with c8aed31fe7c69bb8792dc412870f0fe7c73b3c6c66e703bc64fe7dbd5e456f5f not found: ID does not exist" Jul 18 02:25:08.850242 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.850197 2573 scope.go:117] "RemoveContainer" containerID="479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260" Jul 18 02:25:08.850360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.850269 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f59db8a-d840-431f-af58-fffca9055406-istiod-ca-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:08.850360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.850289 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-istio-token\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:08.850360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.850301 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-data\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:08.850360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.850315 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f59db8a-d840-431f-af58-fffca9055406-istio-podinfo\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:08.850360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.850328 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-istio-envoy\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:08.850360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.850342 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f59db8a-d840-431f-af58-fffca9055406-credential-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:08.850360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.850356 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv2rk\" (UniqueName: \"kubernetes.io/projected/2f59db8a-d840-431f-af58-fffca9055406-kube-api-access-xv2rk\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:08.850570 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:25:08.850434 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260\": container with ID starting with 479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260 not found: ID does not exist" containerID="479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260" Jul 18 02:25:08.850570 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.850460 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260"} err="failed to get container status \"479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260\": rpc error: code = NotFound desc = could not find container \"479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260\": container with ID starting with 479de45180fb054a93d41ed4e45251a5b74b2c35dd53c7753633aba406501260 not found: ID does not exist" Jul 18 02:25:08.854654 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.854630 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d"] Jul 18 02:25:08.859137 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:08.859111 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-5e522e36-gw-openshift-default-5ddfc5c7f9-vz47d"] Jul 18 02:25:10.722500 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:10.722466 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f59db8a-d840-431f-af58-fffca9055406" path="/var/lib/kubelet/pods/2f59db8a-d840-431f-af58-fffca9055406/volumes" Jul 18 02:25:18.885767 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:18.885735 2573 generic.go:356] "Generic (PLEG): container finished" podID="9c63903d-7726-4151-a3c1-2734510dc275" containerID="3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605" exitCode=0 Jul 18 02:25:18.886206 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:18.885808 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" event={"ID":"9c63903d-7726-4151-a3c1-2734510dc275","Type":"ContainerDied","Data":"3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605"} Jul 18 02:25:18.886206 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:18.886194 2573 scope.go:117] "RemoveContainer" containerID="3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605" Jul 18 02:25:18.887144 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:18.887115 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:25:18.887251 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:18.887180 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:25:18.887251 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:18.887212 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:25:19.899691 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:19.899637 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" event={"ID":"9c63903d-7726-4151-a3c1-2734510dc275","Type":"ContainerStarted","Data":"0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5"} Jul 18 02:25:20.554027 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:20.553960 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:20.555216 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:20.555185 2573 patch_prober.go:28] interesting pod/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 18 02:25:20.555345 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:20.555241 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 18 02:25:20.815868 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:20.815784 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:21.553887 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:21.553851 2573 patch_prober.go:28] interesting pod/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 18 02:25:21.554307 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:21.553911 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 18 02:25:22.554481 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:22.554444 2573 patch_prober.go:28] interesting pod/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 18 02:25:22.554873 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:22.554499 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 18 02:25:23.553936 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:23.553898 2573 patch_prober.go:28] interesting pod/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 18 02:25:23.554129 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:23.553952 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 18 02:25:24.557813 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:24.557785 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:24.558851 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:24.558833 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:42.960599 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:42.960561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:47.253303 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:47.253268 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:48.541363 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:48.541329 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c"] Jul 18 02:25:48.541839 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:48.541526 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" containerID="cri-o://0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5" gracePeriod=30 Jul 18 02:25:48.567315 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:48.567265 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:48.835547 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:48.835451 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:49.069781 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.069750 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:49.379087 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.379054 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h"] Jul 18 02:25:49.379379 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.379368 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" Jul 18 02:25:49.379440 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.379381 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" Jul 18 02:25:49.379440 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.379389 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" Jul 18 02:25:49.379440 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.379394 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" Jul 18 02:25:49.379584 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.379454 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" Jul 18 02:25:49.379584 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.379463 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="946948b4-5c6d-44a5-8074-95d0fec4e419" containerName="istio-proxy" Jul 18 02:25:49.381355 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.381338 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.384239 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.384214 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-be63187d-openshift-default-dockercfg-7w9r2\"" Jul 18 02:25:49.398773 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.398753 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h"] Jul 18 02:25:49.404899 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.404869 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:49.509165 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.509132 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istiod-ca-cert\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.509330 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.509172 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-socket\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.509330 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.509192 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-certs\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.509330 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.509223 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-podinfo\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.509330 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.509317 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ztpv\" (UniqueName: \"kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-kube-api-access-7ztpv\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.509533 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.509370 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-data\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.509533 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.509424 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-credential-socket\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.509533 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.509441 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-token\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.509533 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.509454 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-envoy\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.610125 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610091 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istiod-ca-cert\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.610125 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610129 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-socket\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.610649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610154 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-certs\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.610649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610183 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-podinfo\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.610649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610236 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ztpv\" (UniqueName: \"kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-kube-api-access-7ztpv\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.610649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610290 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-data\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.610649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610326 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-credential-socket\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.610649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-token\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.610649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610368 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-envoy\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.610649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610608 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-socket\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.611015 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610790 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-certs\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.611015 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610820 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-data\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.611015 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610847 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-credential-socket\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.611015 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.610903 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istiod-ca-cert\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.612599 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.612572 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-envoy\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.612758 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.612741 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-podinfo\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.619386 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.619359 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-token\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.619709 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.619690 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ztpv\" (UniqueName: \"kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-kube-api-access-7ztpv\") pod \"e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.636667 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.636600 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:49.693094 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.693065 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:49.824936 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.824911 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h"] Jul 18 02:25:49.827433 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:25:49.827402 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a2e4a97_e528_4a2b_a7e2_f9df327bbc23.slice/crio-ab94bf4b93df833dbd22cbc82bcf1b6fec11cf38bf731e52a5d216fd00d5b239 WatchSource:0}: Error finding container ab94bf4b93df833dbd22cbc82bcf1b6fec11cf38bf731e52a5d216fd00d5b239: Status 404 returned error can't find the container with id ab94bf4b93df833dbd22cbc82bcf1b6fec11cf38bf731e52a5d216fd00d5b239 Jul 18 02:25:49.829707 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.829676 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:25:49.829766 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.829739 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:25:49.829807 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:49.829766 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:25:50.048361 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:50.048271 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" event={"ID":"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23","Type":"ContainerStarted","Data":"5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983"} Jul 18 02:25:50.048361 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:50.048323 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" event={"ID":"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23","Type":"ContainerStarted","Data":"ab94bf4b93df833dbd22cbc82bcf1b6fec11cf38bf731e52a5d216fd00d5b239"} Jul 18 02:25:50.071888 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:50.071837 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" podStartSLOduration=1.07182101 podStartE2EDuration="1.07182101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:25:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:25:50.070464921 +0000 UTC m=+1737.858801504" watchObservedRunningTime="2026-07-18 02:25:50.07182101 +0000 UTC m=+1737.860157592" Jul 18 02:25:50.693273 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:50.693232 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:50.698515 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:50.698491 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:51.053093 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:51.053012 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:51.053878 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:51.053861 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:25:51.432140 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:51.432107 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:51.523903 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:51.523858 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:51.599888 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:51.599848 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:51.757055 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:51.756938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:52.253776 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:52.253738 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:52.355605 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:52.355570 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:25:53.787187 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.787160 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:53.950163 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950134 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-envoy\") pod \"9c63903d-7726-4151-a3c1-2734510dc275\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " Jul 18 02:25:53.950163 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950177 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-istio-token\") pod \"9c63903d-7726-4151-a3c1-2734510dc275\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " Jul 18 02:25:53.950429 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950213 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-data\") pod \"9c63903d-7726-4151-a3c1-2734510dc275\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " Jul 18 02:25:53.950429 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950263 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-credential-socket\") pod \"9c63903d-7726-4151-a3c1-2734510dc275\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " Jul 18 02:25:53.950429 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950356 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c63903d-7726-4151-a3c1-2734510dc275-istiod-ca-cert\") pod \"9c63903d-7726-4151-a3c1-2734510dc275\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " Jul 18 02:25:53.950429 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950394 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c63903d-7726-4151-a3c1-2734510dc275-istio-podinfo\") pod \"9c63903d-7726-4151-a3c1-2734510dc275\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " Jul 18 02:25:53.950429 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950420 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-certs\") pod \"9c63903d-7726-4151-a3c1-2734510dc275\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " Jul 18 02:25:53.950693 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950464 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-socket\") pod \"9c63903d-7726-4151-a3c1-2734510dc275\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " Jul 18 02:25:53.950693 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950491 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zzzf\" (UniqueName: \"kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-kube-api-access-5zzzf\") pod \"9c63903d-7726-4151-a3c1-2734510dc275\" (UID: \"9c63903d-7726-4151-a3c1-2734510dc275\") " Jul 18 02:25:53.950693 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950568 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9c63903d-7726-4151-a3c1-2734510dc275" (UID: "9c63903d-7726-4151-a3c1-2734510dc275"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:53.950846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950761 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9c63903d-7726-4151-a3c1-2734510dc275" (UID: "9c63903d-7726-4151-a3c1-2734510dc275"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:53.950846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950794 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c63903d-7726-4151-a3c1-2734510dc275-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9c63903d-7726-4151-a3c1-2734510dc275" (UID: "9c63903d-7726-4151-a3c1-2734510dc275"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:25:53.950846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950838 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-credential-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:53.951041 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.950857 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:53.951097 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.951050 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9c63903d-7726-4151-a3c1-2734510dc275" (UID: "9c63903d-7726-4151-a3c1-2734510dc275"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:53.951364 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.951339 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-data" (OuterVolumeSpecName: "istio-data") pod "9c63903d-7726-4151-a3c1-2734510dc275" (UID: "9c63903d-7726-4151-a3c1-2734510dc275"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:53.952489 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.952465 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-istio-token" (OuterVolumeSpecName: "istio-token") pod "9c63903d-7726-4151-a3c1-2734510dc275" (UID: "9c63903d-7726-4151-a3c1-2734510dc275"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:25:53.952596 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.952568 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c63903d-7726-4151-a3c1-2734510dc275-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9c63903d-7726-4151-a3c1-2734510dc275" (UID: "9c63903d-7726-4151-a3c1-2734510dc275"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:25:53.952730 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.952711 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9c63903d-7726-4151-a3c1-2734510dc275" (UID: "9c63903d-7726-4151-a3c1-2734510dc275"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:53.952921 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:53.952903 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-kube-api-access-5zzzf" (OuterVolumeSpecName: "kube-api-access-5zzzf") pod "9c63903d-7726-4151-a3c1-2734510dc275" (UID: "9c63903d-7726-4151-a3c1-2734510dc275"). InnerVolumeSpecName "kube-api-access-5zzzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:25:54.051656 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.051621 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c63903d-7726-4151-a3c1-2734510dc275-istiod-ca-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:54.051656 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.051649 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c63903d-7726-4151-a3c1-2734510dc275-istio-podinfo\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:54.051656 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.051658 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-workload-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:54.051894 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.051666 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zzzf\" (UniqueName: \"kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-kube-api-access-5zzzf\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:54.051894 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.051677 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-envoy\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:54.051894 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.051685 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c63903d-7726-4151-a3c1-2734510dc275-istio-token\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:54.051894 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.051693 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c63903d-7726-4151-a3c1-2734510dc275-istio-data\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:25:54.067487 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.067285 2573 generic.go:356] "Generic (PLEG): container finished" podID="9c63903d-7726-4151-a3c1-2734510dc275" containerID="0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5" exitCode=0 Jul 18 02:25:54.067487 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.067345 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" Jul 18 02:25:54.067487 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.067351 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" event={"ID":"9c63903d-7726-4151-a3c1-2734510dc275","Type":"ContainerDied","Data":"0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5"} Jul 18 02:25:54.067487 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.067417 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c" event={"ID":"9c63903d-7726-4151-a3c1-2734510dc275","Type":"ContainerDied","Data":"c6264f03c1733a7930d9dd5b231d20147e66b697143ed3bbc6bb39107b5ef6a5"} Jul 18 02:25:54.067487 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.067450 2573 scope.go:117] "RemoveContainer" containerID="0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5" Jul 18 02:25:54.075931 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.075912 2573 scope.go:117] "RemoveContainer" containerID="3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605" Jul 18 02:25:54.083321 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.083305 2573 scope.go:117] "RemoveContainer" containerID="0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5" Jul 18 02:25:54.083558 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:25:54.083541 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5\": container with ID starting with 0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5 not found: ID does not exist" containerID="0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5" Jul 18 02:25:54.083607 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.083566 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5"} err="failed to get container status \"0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5\": rpc error: code = NotFound desc = could not find container \"0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5\": container with ID starting with 0eec96b82145e8c1c535fc795127c44671a54d6270352fbae2d04a1694a909d5 not found: ID does not exist" Jul 18 02:25:54.083607 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.083582 2573 scope.go:117] "RemoveContainer" containerID="3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605" Jul 18 02:25:54.083839 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:25:54.083821 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605\": container with ID starting with 3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605 not found: ID does not exist" containerID="3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605" Jul 18 02:25:54.083894 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.083844 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605"} err="failed to get container status \"3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605\": rpc error: code = NotFound desc = could not find container \"3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605\": container with ID starting with 3456d9af55b8b2800c059d401b318d1cec05209aa7610b3858e7cec8a525b605 not found: ID does not exist" Jul 18 02:25:54.090247 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.090224 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c"] Jul 18 02:25:54.093659 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.093638 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-eaf8d750-openshift-default-86cfdbd975-nbg8c"] Jul 18 02:25:54.722643 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:25:54.722610 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c63903d-7726-4151-a3c1-2734510dc275" path="/var/lib/kubelet/pods/9c63903d-7726-4151-a3c1-2734510dc275/volumes" Jul 18 02:26:05.156561 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:05.156520 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:05.247645 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:05.247598 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:06.055264 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:06.055228 2573 patch_prober.go:28] interesting pod/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 18 02:26:06.055441 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:06.055296 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 18 02:26:06.133374 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:06.133342 2573 generic.go:356] "Generic (PLEG): container finished" podID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerID="5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983" exitCode=0 Jul 18 02:26:06.133549 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:06.133424 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" event={"ID":"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23","Type":"ContainerDied","Data":"5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983"} Jul 18 02:26:06.133860 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:06.133843 2573 scope.go:117] "RemoveContainer" containerID="5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983" Jul 18 02:26:06.134858 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:06.134828 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:26:06.134913 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:06.134899 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:26:06.134946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:06.134930 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:26:07.140303 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:07.140267 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" event={"ID":"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23","Type":"ContainerStarted","Data":"3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9"} Jul 18 02:26:07.693639 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:07.693600 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:26:07.693907 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:07.693877 2573 patch_prober.go:28] interesting pod/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 18 02:26:07.694003 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:07.693941 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 18 02:26:08.703800 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:08.698425 2573 patch_prober.go:28] interesting pod/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 18 02:26:08.703800 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:08.698515 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 18 02:26:09.693654 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:09.693613 2573 patch_prober.go:28] interesting pod/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 18 02:26:09.694009 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:09.693955 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 18 02:26:10.693734 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:10.693687 2573 patch_prober.go:28] interesting pod/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 18 02:26:10.694262 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:10.693771 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 18 02:26:11.697763 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:11.697733 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:26:11.698250 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:11.698065 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:26:11.698666 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:11.698646 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:26:26.924925 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:26.924892 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:31.433328 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:31.433287 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:31.581348 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:31.581308 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h"] Jul 18 02:26:31.581555 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:31.581526 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" containerID="cri-o://3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9" gracePeriod=30 Jul 18 02:26:31.604720 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:31.604675 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:31.861329 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:31.861228 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:32.372175 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:32.372139 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:32.470141 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:32.469912 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:32.571568 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:32.571536 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:34.089300 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:34.089267 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:36.871807 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.871775 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:26:36.938735 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.938697 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-certs\") pod \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " Jul 18 02:26:36.938735 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.938738 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-data\") pod \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " Jul 18 02:26:36.938992 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.938753 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-credential-socket\") pod \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " Jul 18 02:26:36.938992 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.938774 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-envoy\") pod \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " Jul 18 02:26:36.938992 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.938800 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-token\") pod \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " Jul 18 02:26:36.938992 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.938856 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-socket\") pod \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " Jul 18 02:26:36.938992 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.938914 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istiod-ca-cert\") pod \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " Jul 18 02:26:36.939238 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.939019 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-podinfo\") pod \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " Jul 18 02:26:36.939238 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.939028 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" (UID: "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:36.939238 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.939059 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" (UID: "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:36.939238 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.939045 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ztpv\" (UniqueName: \"kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-kube-api-access-7ztpv\") pod \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\" (UID: \"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23\") " Jul 18 02:26:36.939238 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.939187 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" (UID: "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:36.939478 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.939405 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:26:36.939478 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.939427 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-credential-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:26:36.939478 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.939441 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-workload-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:26:36.939649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.939573 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" (UID: "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:26:36.939649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.939603 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" (UID: "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:36.941324 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.941302 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" (UID: "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:36.941562 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.941542 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" (UID: "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:26:36.941634 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.941560 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-kube-api-access-7ztpv" (OuterVolumeSpecName: "kube-api-access-7ztpv") pod "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" (UID: "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23"). InnerVolumeSpecName "kube-api-access-7ztpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:26:36.941729 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:36.941710 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" (UID: "4a2e4a97-e528-4a2b-a7e2-f9df327bbc23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:26:37.040492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.040455 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-podinfo\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:26:37.040492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.040487 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ztpv\" (UniqueName: \"kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-kube-api-access-7ztpv\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:26:37.040492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.040496 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-data\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:26:37.041045 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.040506 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-envoy\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:26:37.041045 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.040513 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istio-token\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:26:37.041045 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.040521 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23-istiod-ca-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:26:37.281295 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.281262 2573 generic.go:356] "Generic (PLEG): container finished" podID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerID="3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9" exitCode=0 Jul 18 02:26:37.281501 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.281312 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" Jul 18 02:26:37.281501 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.281347 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" event={"ID":"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23","Type":"ContainerDied","Data":"3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9"} Jul 18 02:26:37.281501 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.281402 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h" event={"ID":"4a2e4a97-e528-4a2b-a7e2-f9df327bbc23","Type":"ContainerDied","Data":"ab94bf4b93df833dbd22cbc82bcf1b6fec11cf38bf731e52a5d216fd00d5b239"} Jul 18 02:26:37.281501 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.281424 2573 scope.go:117] "RemoveContainer" containerID="3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9" Jul 18 02:26:37.290518 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.290497 2573 scope.go:117] "RemoveContainer" containerID="5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983" Jul 18 02:26:37.298020 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.298002 2573 scope.go:117] "RemoveContainer" containerID="3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9" Jul 18 02:26:37.298261 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:26:37.298240 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9\": container with ID starting with 3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9 not found: ID does not exist" containerID="3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9" Jul 18 02:26:37.298341 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.298272 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9"} err="failed to get container status \"3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9\": rpc error: code = NotFound desc = could not find container \"3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9\": container with ID starting with 3a10b7e77566521c15290de35f8bb19e4f3059a18de895117705a847c751c3a9 not found: ID does not exist" Jul 18 02:26:37.298341 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.298296 2573 scope.go:117] "RemoveContainer" containerID="5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983" Jul 18 02:26:37.298546 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:26:37.298529 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983\": container with ID starting with 5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983 not found: ID does not exist" containerID="5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983" Jul 18 02:26:37.298586 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.298554 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983"} err="failed to get container status \"5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983\": rpc error: code = NotFound desc = could not find container \"5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983\": container with ID starting with 5dacd5207d84c4f8cd93e674e1605dece45571ec132bae251cfc116bea6c1983 not found: ID does not exist" Jul 18 02:26:37.302523 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.302503 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h"] Jul 18 02:26:37.306627 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:37.306605 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-be63187d-openshift-default-845dfd57ff-47c8h"] Jul 18 02:26:38.722493 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:38.722460 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" path="/var/lib/kubelet/pods/4a2e4a97-e528-4a2b-a7e2-f9df327bbc23/volumes" Jul 18 02:26:40.172099 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:40.172071 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:40.258768 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:40.258730 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:40.371720 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:40.371688 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:40.471831 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:40.471743 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:40.945134 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:40.945091 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:52.757705 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:52.757679 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:26:52.762961 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:52.762943 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:26:53.812411 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:53.812369 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:26:53.913374 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:26:53.913341 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:06.990224 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:06.990186 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:13.694353 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:13.694314 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:13.798915 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:13.798870 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:13.894903 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:13.894869 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:29.868586 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:29.868547 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:36.463517 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:36.463485 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:36.576349 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:36.576321 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:36.663530 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:36.663493 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:47.212921 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:47.212878 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:47.672376 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:47.672344 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:47.974911 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:47.974828 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:55.345827 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:55.345776 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:55.448917 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:55.448879 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:55.556635 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:55.556603 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:55.644321 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:55.644290 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:56.523702 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:56.523672 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:56.712597 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:56.712563 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:27:58.332626 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:27:58.332591 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:04.403306 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:04.403272 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:04.471465 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:04.471432 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:05.112277 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:05.112246 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:11.590999 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:11.590937 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:11.690895 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:11.690858 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:11.831608 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:11.831568 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:12.246905 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.246874 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp"] Jul 18 02:28:12.247353 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247338 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" Jul 18 02:28:12.247402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247356 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" Jul 18 02:28:12.247402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247391 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" Jul 18 02:28:12.247402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247400 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" Jul 18 02:28:12.247492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247410 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" Jul 18 02:28:12.247492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247418 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" Jul 18 02:28:12.247492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247488 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" Jul 18 02:28:12.247589 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247500 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c63903d-7726-4151-a3c1-2734510dc275" containerName="istio-proxy" Jul 18 02:28:12.247589 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247508 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f59db8a-d840-431f-af58-fffca9055406" containerName="istio-proxy" Jul 18 02:28:12.247589 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247517 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" Jul 18 02:28:12.247589 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.247527 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" Jul 18 02:28:12.250795 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.250772 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.254157 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.254139 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2bf983eb-openshift-default-dockercfg-775wf\"" Jul 18 02:28:12.278861 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.278811 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp"] Jul 18 02:28:12.285822 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.285773 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:12.348244 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.348183 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-token\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.348509 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.348300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-podinfo\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.348509 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.348414 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxv2n\" (UniqueName: \"kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-kube-api-access-vxv2n\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.348509 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.348498 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-data\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.348655 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.348534 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-credential-socket\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.348655 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.348597 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-certs\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.348655 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.348651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-envoy\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.348771 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.348680 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istiod-ca-cert\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.348771 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.348736 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-socket\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.449860 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.449829 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-podinfo\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.449887 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxv2n\" (UniqueName: \"kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-kube-api-access-vxv2n\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.449928 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-data\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.449958 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-credential-socket\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.449999 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-certs\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.450033 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-envoy\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.450064 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istiod-ca-cert\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.450099 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-socket\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450451 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.450147 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-token\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450451 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.450380 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-data\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450540 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.450457 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-certs\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450644 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.450621 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-socket\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450739 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.450719 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-credential-socket\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.450839 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.450815 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istiod-ca-cert\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.452330 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.452305 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-envoy\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.452517 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.452498 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-podinfo\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.458595 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.458569 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-token\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.458694 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.458636 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxv2n\" (UniqueName: \"kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-kube-api-access-vxv2n\") pod \"e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.489809 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.489762 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:12.562282 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.562192 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:12.732759 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.732735 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp"] Jul 18 02:28:12.736219 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:28:12.736181 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode44926cc_0b59_479f_87b6_1fd1bfb4d5c3.slice/crio-17a195f127603a5c555ed0e9855d0883f5a5ddab0679bf0009e31da1c62a133f WatchSource:0}: Error finding container 17a195f127603a5c555ed0e9855d0883f5a5ddab0679bf0009e31da1c62a133f: Status 404 returned error can't find the container with id 17a195f127603a5c555ed0e9855d0883f5a5ddab0679bf0009e31da1c62a133f Jul 18 02:28:12.738652 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.738576 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:28:12.738777 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.738665 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:28:12.738777 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:12.738708 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:28:13.744697 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:13.744660 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" event={"ID":"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3","Type":"ContainerStarted","Data":"6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1"} Jul 18 02:28:13.745241 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:13.744711 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" event={"ID":"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3","Type":"ContainerStarted","Data":"17a195f127603a5c555ed0e9855d0883f5a5ddab0679bf0009e31da1c62a133f"} Jul 18 02:28:13.768219 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:13.768171 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" podStartSLOduration=1.768154365 podStartE2EDuration="1.768154365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:28:13.766347998 +0000 UTC m=+1881.554684602" watchObservedRunningTime="2026-07-18 02:28:13.768154365 +0000 UTC m=+1881.556490945" Jul 18 02:28:14.563078 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:14.563041 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:14.567848 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:14.567824 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:14.749866 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:14.749826 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:14.750826 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:14.750804 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:15.714247 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:15.714139 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:15.804990 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:15.804941 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:15.899760 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:15.899727 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:16.117931 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:16.117837 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:16.373251 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:16.373214 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:16.473581 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:16.473542 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:20.195775 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:20.195742 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:20.877784 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:20.877751 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:20.977405 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:20.977360 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:21.322359 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:21.322273 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:27.459602 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:27.459558 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:27.560301 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:27.560264 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:28.718588 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:28.718552 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:29.752106 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:29.752011 2573 patch_prober.go:28] interesting pod/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 18 02:28:29.752106 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:29.752071 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 18 02:28:29.832523 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:29.832490 2573 generic.go:356] "Generic (PLEG): container finished" podID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerID="6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1" exitCode=0 Jul 18 02:28:29.832674 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:29.832561 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" event={"ID":"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3","Type":"ContainerDied","Data":"6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1"} Jul 18 02:28:29.832897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:29.832885 2573 scope.go:117] "RemoveContainer" containerID="6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1" Jul 18 02:28:29.833879 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:29.833854 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:28:29.833936 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:29.833919 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:28:29.833988 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:29.833949 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:28:30.840451 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:30.840415 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" event={"ID":"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3","Type":"ContainerStarted","Data":"e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448"} Jul 18 02:28:31.563008 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:31.562957 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:31.563178 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:31.563115 2573 patch_prober.go:28] interesting pod/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 18 02:28:31.563178 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:31.563154 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 18 02:28:32.562858 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:32.562827 2573 patch_prober.go:28] interesting pod/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 18 02:28:32.563241 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:32.562884 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 18 02:28:33.563465 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:33.563423 2573 patch_prober.go:28] interesting pod/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 18 02:28:33.563934 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:33.563482 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 18 02:28:34.563134 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:34.563099 2573 patch_prober.go:28] interesting pod/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 18 02:28:34.563305 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:34.563160 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 18 02:28:35.567079 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:35.567048 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:35.567439 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:35.567413 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:35.568268 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:35.568248 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:38.560999 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:38.560955 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:42.992253 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:42.992218 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:43.389042 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:43.389008 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:49.244015 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:49.243963 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:49.340361 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:49.340328 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:49.506279 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:49.506189 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp"] Jul 18 02:28:49.506441 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:49.506404 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" containerID="cri-o://e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448" gracePeriod=30 Jul 18 02:28:49.530322 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:49.530283 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:49.810925 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:49.810831 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:49.944788 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:49.944743 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:50.046098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:50.046059 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:50.196063 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:50.196026 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:50.570273 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:50.570190 2573 patch_prober.go:28] interesting pod/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 02:28:50.570273 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:50.570254 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 02:28:54.748418 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.748392 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:54.811831 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.811750 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-data\") pod \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " Jul 18 02:28:54.811831 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.811810 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-token\") pod \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " Jul 18 02:28:54.811831 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.811825 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istiod-ca-cert\") pod \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " Jul 18 02:28:54.812100 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.811882 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-certs\") pod \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " Jul 18 02:28:54.812100 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.811919 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxv2n\" (UniqueName: \"kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-kube-api-access-vxv2n\") pod \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " Jul 18 02:28:54.812100 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.811966 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-socket\") pod \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " Jul 18 02:28:54.812100 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.812019 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-podinfo\") pod \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " Jul 18 02:28:54.812100 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.812047 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-credential-socket\") pod \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " Jul 18 02:28:54.812100 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.812069 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-envoy\") pod \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\" (UID: \"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3\") " Jul 18 02:28:54.812396 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.812199 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" (UID: "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:28:54.812396 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.812273 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" (UID: "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:28:54.812396 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.812335 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-data" (OuterVolumeSpecName: "istio-data") pod "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" (UID: "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:28:54.813058 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.812773 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:28:54.813058 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.812797 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-data\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:28:54.813058 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.812810 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istiod-ca-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:28:54.813058 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.813017 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" (UID: "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:28:54.813371 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.813350 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" (UID: "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:28:54.814301 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.814272 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-token" (OuterVolumeSpecName: "istio-token") pod "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" (UID: "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:28:54.814405 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.814292 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-kube-api-access-vxv2n" (OuterVolumeSpecName: "kube-api-access-vxv2n") pod "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" (UID: "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3"). InnerVolumeSpecName "kube-api-access-vxv2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:28:54.814859 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.814839 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" (UID: "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:28:54.814954 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.814854 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" (UID: "e44926cc-0b59-479f-87b6-1fd1bfb4d5c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:28:54.913275 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.913228 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-credential-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:28:54.913275 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.913271 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-envoy\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:28:54.913275 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.913281 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-token\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:28:54.913275 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.913290 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxv2n\" (UniqueName: \"kubernetes.io/projected/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-kube-api-access-vxv2n\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:28:54.913546 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.913298 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-workload-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:28:54.913546 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.913306 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3-istio-podinfo\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:28:54.955578 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.955546 2573 generic.go:356] "Generic (PLEG): container finished" podID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerID="e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448" exitCode=0 Jul 18 02:28:54.955741 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.955594 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" Jul 18 02:28:54.955741 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.955629 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" event={"ID":"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3","Type":"ContainerDied","Data":"e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448"} Jul 18 02:28:54.955741 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.955682 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp" event={"ID":"e44926cc-0b59-479f-87b6-1fd1bfb4d5c3","Type":"ContainerDied","Data":"17a195f127603a5c555ed0e9855d0883f5a5ddab0679bf0009e31da1c62a133f"} Jul 18 02:28:54.955741 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.955722 2573 scope.go:117] "RemoveContainer" containerID="e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448" Jul 18 02:28:54.965328 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.965313 2573 scope.go:117] "RemoveContainer" containerID="6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1" Jul 18 02:28:54.972797 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.972782 2573 scope.go:117] "RemoveContainer" containerID="e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448" Jul 18 02:28:54.973048 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:28:54.973031 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448\": container with ID starting with e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448 not found: ID does not exist" containerID="e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448" Jul 18 02:28:54.973120 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.973055 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448"} err="failed to get container status \"e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448\": rpc error: code = NotFound desc = could not find container \"e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448\": container with ID starting with e141977878378fde760d6ca87801e1cda74f06a315b21ffbbd3a3cb6ee8c6448 not found: ID does not exist" Jul 18 02:28:54.973120 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.973072 2573 scope.go:117] "RemoveContainer" containerID="6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1" Jul 18 02:28:54.973276 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:28:54.973262 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1\": container with ID starting with 6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1 not found: ID does not exist" containerID="6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1" Jul 18 02:28:54.973318 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.973280 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1"} err="failed to get container status \"6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1\": rpc error: code = NotFound desc = could not find container \"6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1\": container with ID starting with 6e4461eead2f7b8ce1c49bc0d77383e84c2d5b4c1fa1b9aa1bea1dc983e09eb1 not found: ID does not exist" Jul 18 02:28:54.978814 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.978794 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp"] Jul 18 02:28:54.982040 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:54.982017 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2bf983eb-openshift-default-56f59d867f-x56bp"] Jul 18 02:28:55.963754 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:55.963722 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:56.077595 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:56.077564 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:56.164630 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:56.164601 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:56.349897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:56.349808 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:28:56.722212 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:28:56.722178 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" path="/var/lib/kubelet/pods/e44926cc-0b59-479f-87b6-1fd1bfb4d5c3/volumes" Jul 18 02:29:02.367756 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:02.367723 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:02.468739 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:02.468704 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:09.387846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.387808 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:09.667439 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.667358 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr"] Jul 18 02:29:09.667687 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.667675 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" Jul 18 02:29:09.667728 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.667689 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a2e4a97-e528-4a2b-a7e2-f9df327bbc23" containerName="istio-proxy" Jul 18 02:29:09.667728 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.667704 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" Jul 18 02:29:09.667728 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.667709 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" Jul 18 02:29:09.667728 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.667721 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" Jul 18 02:29:09.667728 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.667726 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" Jul 18 02:29:09.667877 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.667780 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" Jul 18 02:29:09.670927 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.670907 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.674649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.674628 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-327441da-openshift-default-dockercfg-tq45l\"" Jul 18 02:29:09.699315 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.699286 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr"] Jul 18 02:29:09.705535 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.705506 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:09.738676 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.738640 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-credential-socket\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.738828 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.738685 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-podinfo\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.738828 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.738748 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvnkt\" (UniqueName: \"kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-kube-api-access-cvnkt\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.738828 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.738789 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-certs\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.738828 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.738804 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-token\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.738828 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.738826 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-data\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.739164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.738939 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-socket\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.739164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.739011 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-envoy\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.739164 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.739043 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istiod-ca-cert\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.788636 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.788600 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:09.840524 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.840491 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istiod-ca-cert\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.840698 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.840538 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-credential-socket\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.840698 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.840566 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-podinfo\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.840698 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.840594 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvnkt\" (UniqueName: \"kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-kube-api-access-cvnkt\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.840698 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.840627 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-certs\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.840698 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.840644 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-token\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.840698 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.840674 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-data\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.841022 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.840713 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-socket\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.841022 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.840740 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-envoy\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.841150 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.841088 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-data\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.841210 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.841175 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-credential-socket\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.841319 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.841290 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-certs\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.841438 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.841391 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-socket\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.841438 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.841397 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istiod-ca-cert\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.842913 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.842891 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-envoy\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.843128 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.843113 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-podinfo\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.850171 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.850149 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-token\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.850428 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.850407 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvnkt\" (UniqueName: \"kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-kube-api-access-cvnkt\") pod \"e2e-mt-327441da-openshift-default-678fb67f47-ltsgr\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:09.981388 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:09.981291 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:10.131437 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:10.131405 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr"] Jul 18 02:29:10.134792 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:29:10.134771 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee008ff9_50cb_4e45_82ff_7f90a75d1ab4.slice/crio-848e9192f0411288b0ed7983b1a3b1827903c2354449b4fa4b64dffe4d1bf103 WatchSource:0}: Error finding container 848e9192f0411288b0ed7983b1a3b1827903c2354449b4fa4b64dffe4d1bf103: Status 404 returned error can't find the container with id 848e9192f0411288b0ed7983b1a3b1827903c2354449b4fa4b64dffe4d1bf103 Jul 18 02:29:10.137217 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:10.137187 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:29:10.137278 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:10.137251 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:29:10.137318 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:10.137277 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:29:11.035130 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:11.035094 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" event={"ID":"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4","Type":"ContainerStarted","Data":"98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348"} Jul 18 02:29:11.035524 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:11.035146 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" event={"ID":"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4","Type":"ContainerStarted","Data":"848e9192f0411288b0ed7983b1a3b1827903c2354449b4fa4b64dffe4d1bf103"} Jul 18 02:29:11.059077 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:11.059031 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" podStartSLOduration=2.059015769 podStartE2EDuration="2.059015769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:29:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:29:11.056215022 +0000 UTC m=+1938.844551602" watchObservedRunningTime="2026-07-18 02:29:11.059015769 +0000 UTC m=+1938.847352350" Jul 18 02:29:11.982046 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:11.982006 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:11.986473 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:11.986448 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:12.040011 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:12.039967 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:12.040965 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:12.040946 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:13.009040 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:13.009009 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:13.104822 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:13.104771 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:13.185206 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:13.185173 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:13.330664 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:13.330573 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:13.686186 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:13.686151 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:13.788206 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:13.788158 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:17.561891 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:17.561859 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:29:27.041348 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:27.041256 2573 patch_prober.go:28] interesting pod/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 18 02:29:27.041348 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:27.041308 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 18 02:29:27.122797 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:27.122766 2573 generic.go:356] "Generic (PLEG): container finished" podID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerID="98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348" exitCode=0 Jul 18 02:29:27.122961 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:27.122814 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" event={"ID":"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4","Type":"ContainerDied","Data":"98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348"} Jul 18 02:29:27.123197 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:27.123184 2573 scope.go:117] "RemoveContainer" containerID="98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348" Jul 18 02:29:27.124135 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:27.124105 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:29:27.124244 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:27.124186 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:29:27.124308 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:27.124244 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:29:28.129791 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:28.129756 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" event={"ID":"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4","Type":"ContainerStarted","Data":"45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2"} Jul 18 02:29:28.982077 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:28.982034 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:28.982414 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:28.982386 2573 patch_prober.go:28] interesting pod/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 18 02:29:28.982520 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:28.982449 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 18 02:29:29.981940 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:29.981903 2573 patch_prober.go:28] interesting pod/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 18 02:29:29.982533 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:29.982502 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 18 02:29:30.982366 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:30.982325 2573 patch_prober.go:28] interesting pod/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 18 02:29:30.982844 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:30.982391 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 18 02:29:31.985488 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:31.985460 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:31.985843 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:31.985810 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:31.986417 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:31.986402 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:29:34.350728 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:29:34.350700 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:30:11.777850 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:30:11.777817 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:30:16.322443 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:30:16.322412 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:30:18.692909 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:30:18.692874 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:30:23.767148 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:30:23.767090 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:30:59.831638 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:30:59.831581 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:00.847551 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:00.847515 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:06.526243 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:06.526177 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:20.693104 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:20.693021 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:20.792790 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:20.792749 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:20.945565 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:20.945486 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr"] Jul 18 02:31:20.945785 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:20.945761 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" containerID="cri-o://45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2" gracePeriod=30 Jul 18 02:31:20.961867 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:20.961834 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:21.203705 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:21.203627 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:22.300438 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:22.300407 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:22.399580 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:22.399544 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:22.500026 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:22.499925 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:23.296098 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:23.296048 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:23.399215 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:23.399180 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:23.496295 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:23.496244 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:24.880016 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:24.879981 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:26.223225 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.223191 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:31:26.331492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331456 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-certs\") pod \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " Jul 18 02:31:26.331661 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331527 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-credential-socket\") pod \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " Jul 18 02:31:26.331661 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331551 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-data\") pod \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " Jul 18 02:31:26.331661 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331577 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istiod-ca-cert\") pod \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " Jul 18 02:31:26.331661 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331611 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-envoy\") pod \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " Jul 18 02:31:26.331661 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331639 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-podinfo\") pod \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " Jul 18 02:31:26.331901 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331680 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-token\") pod \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " Jul 18 02:31:26.331901 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331734 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvnkt\" (UniqueName: \"kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-kube-api-access-cvnkt\") pod \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " Jul 18 02:31:26.331901 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331738 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" (UID: "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:31:26.331901 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331765 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-socket\") pod \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\" (UID: \"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4\") " Jul 18 02:31:26.331901 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.331835 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" (UID: "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:31:26.332171 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.332073 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:31:26.332171 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.332092 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-credential-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:31:26.332286 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.332223 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" (UID: "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:31:26.332537 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.332476 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" (UID: "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:31:26.332537 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.332509 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" (UID: "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:31:26.334380 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.334346 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-kube-api-access-cvnkt" (OuterVolumeSpecName: "kube-api-access-cvnkt") pod "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" (UID: "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4"). InnerVolumeSpecName "kube-api-access-cvnkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:31:26.334498 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.334451 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" (UID: "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:31:26.334498 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.334469 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" (UID: "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:31:26.334619 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.334564 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" (UID: "ee008ff9-50cb-4e45-82ff-7f90a75d1ab4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:31:26.433365 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.433331 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-data\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:31:26.433365 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.433367 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istiod-ca-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:31:26.433562 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.433380 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-envoy\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:31:26.433562 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.433392 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-podinfo\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:31:26.433562 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.433404 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-istio-token\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:31:26.433562 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.433417 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvnkt\" (UniqueName: \"kubernetes.io/projected/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-kube-api-access-cvnkt\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:31:26.433562 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.433430 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4-workload-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:31:26.713694 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.713655 2573 generic.go:356] "Generic (PLEG): container finished" podID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerID="45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2" exitCode=0 Jul 18 02:31:26.713901 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.713739 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" Jul 18 02:31:26.713901 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.713739 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" event={"ID":"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4","Type":"ContainerDied","Data":"45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2"} Jul 18 02:31:26.714039 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.713940 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr" event={"ID":"ee008ff9-50cb-4e45-82ff-7f90a75d1ab4","Type":"ContainerDied","Data":"848e9192f0411288b0ed7983b1a3b1827903c2354449b4fa4b64dffe4d1bf103"} Jul 18 02:31:26.714039 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.713990 2573 scope.go:117] "RemoveContainer" containerID="45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2" Jul 18 02:31:26.724125 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.724088 2573 scope.go:117] "RemoveContainer" containerID="98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348" Jul 18 02:31:26.733282 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.733262 2573 scope.go:117] "RemoveContainer" containerID="45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2" Jul 18 02:31:26.733511 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:31:26.733492 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2\": container with ID starting with 45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2 not found: ID does not exist" containerID="45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2" Jul 18 02:31:26.733594 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.733517 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2"} err="failed to get container status \"45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2\": rpc error: code = NotFound desc = could not find container \"45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2\": container with ID starting with 45bfc261260652346c9cc849d130f9f10bcd78cd193c4635f69dccdc58f152f2 not found: ID does not exist" Jul 18 02:31:26.733594 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.733533 2573 scope.go:117] "RemoveContainer" containerID="98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348" Jul 18 02:31:26.733741 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:31:26.733727 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348\": container with ID starting with 98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348 not found: ID does not exist" containerID="98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348" Jul 18 02:31:26.733804 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.733741 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348"} err="failed to get container status \"98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348\": rpc error: code = NotFound desc = could not find container \"98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348\": container with ID starting with 98ab47c36d0420c33cb37b850c6363620551302df19944714f843fc6c03c8348 not found: ID does not exist" Jul 18 02:31:26.741245 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.741218 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr"] Jul 18 02:31:26.749465 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:26.749431 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-327441da-openshift-default-678fb67f47-ltsgr"] Jul 18 02:31:28.722026 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:28.721986 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" path="/var/lib/kubelet/pods/ee008ff9-50cb-4e45-82ff-7f90a75d1ab4/volumes" Jul 18 02:31:30.825927 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:30.825891 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:30.925929 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:30.925892 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:31.539961 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:31.539927 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:37.683946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:37.683905 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:37.786919 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:37.786883 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:37.884712 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:37.884678 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:48.415932 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:48.415898 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:48.743599 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:48.743517 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:52.788124 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:52.788099 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:31:52.793926 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:52.793905 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:31:54.566086 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:54.565989 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:31:54.666960 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:31:54.666926 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:09.720448 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:09.720415 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:15.732736 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:15.732701 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:15.834272 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:15.834239 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:16.085680 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:16.085596 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:16.540600 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:16.540569 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:16.673181 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:16.673145 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:21.698189 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:21.698150 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:21.787501 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:21.787467 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:21.859518 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:21.859476 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:22.059814 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:22.059717 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:22.162039 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:22.162009 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:22.262796 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:22.262760 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:22.378721 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:22.378682 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:22.486339 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:22.486294 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:26.166599 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.166566 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:26.480488 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.480400 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:26.753957 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.753867 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k"] Jul 18 02:32:26.754259 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.754243 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" Jul 18 02:32:26.754259 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.754258 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" Jul 18 02:32:26.754399 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.754266 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" Jul 18 02:32:26.754399 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.754272 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" Jul 18 02:32:26.754399 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.754334 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" Jul 18 02:32:26.754399 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.754343 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e44926cc-0b59-479f-87b6-1fd1bfb4d5c3" containerName="istio-proxy" Jul 18 02:32:26.757331 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.757308 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.759704 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.759688 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a45ea2-openshift-default-dockercfg-dkvbk\"" Jul 18 02:32:26.767618 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.767595 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k"] Jul 18 02:32:26.781126 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.781088 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:26.863867 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.863827 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-credential-socket\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.864115 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.863995 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-token\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.864115 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.864022 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-podinfo\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.864115 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.864051 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istiod-ca-cert\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.864115 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.864077 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-envoy\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.864414 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.864128 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-data\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.864414 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.864160 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-certs\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.864414 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.864188 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-socket\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.864414 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.864221 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thmxz\" (UniqueName: \"kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-kube-api-access-thmxz\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.937693 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.937661 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:26.964748 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.964700 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thmxz\" (UniqueName: \"kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-kube-api-access-thmxz\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.964748 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.964758 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-credential-socket\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.964809 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-token\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.964826 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-podinfo\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.964865 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istiod-ca-cert\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.964892 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-envoy\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.964935 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-data\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.964965 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-certs\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.965018 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-socket\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965392 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.965350 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-credential-socket\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965461 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.965401 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-socket\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965513 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.965465 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-certs\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965669 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.965649 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-data\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.965732 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.965717 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istiod-ca-cert\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.967269 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.967242 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-envoy\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.967410 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.967395 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-podinfo\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.971952 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.971930 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-token\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:26.972112 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:26.972094 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thmxz\" (UniqueName: \"kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-kube-api-access-thmxz\") pod \"e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:27.068443 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:27.068332 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:27.207795 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:27.207769 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k"] Jul 18 02:32:27.209646 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:32:27.209613 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f0efc41_4f6e_43a8_841c_b2a1a3af81c0.slice/crio-c7b13c0fdbf78e27f877f756be5e437d2c5be1d5ad867ab1599d8561ccd0e375 WatchSource:0}: Error finding container c7b13c0fdbf78e27f877f756be5e437d2c5be1d5ad867ab1599d8561ccd0e375: Status 404 returned error can't find the container with id c7b13c0fdbf78e27f877f756be5e437d2c5be1d5ad867ab1599d8561ccd0e375 Jul 18 02:32:27.211821 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:27.211777 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:32:27.211904 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:27.211869 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:32:27.211942 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:27.211913 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:32:27.999893 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:27.999855 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" event={"ID":"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0","Type":"ContainerStarted","Data":"3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865"} Jul 18 02:32:28.000104 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:27.999919 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" event={"ID":"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0","Type":"ContainerStarted","Data":"c7b13c0fdbf78e27f877f756be5e437d2c5be1d5ad867ab1599d8561ccd0e375"} Jul 18 02:32:28.020978 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:28.020925 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" podStartSLOduration=2.020907629 podStartE2EDuration="2.020907629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:32:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:32:28.018473903 +0000 UTC m=+2135.806810486" watchObservedRunningTime="2026-07-18 02:32:28.020907629 +0000 UTC m=+2135.809244211" Jul 18 02:32:28.069016 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:28.068958 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:28.074133 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:28.074108 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:29.005008 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:29.004962 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:29.005995 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:29.005951 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:30.041380 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:30.041349 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:30.125532 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:30.125494 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:30.220552 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:30.220523 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:30.332407 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:30.332316 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:30.740768 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:30.740732 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:30.811959 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:30.811924 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:34.380906 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:34.380868 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:39.701288 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:39.701240 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:42.067732 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:42.067638 2573 generic.go:356] "Generic (PLEG): container finished" podID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerID="3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865" exitCode=0 Jul 18 02:32:42.067732 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:42.067701 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" event={"ID":"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0","Type":"ContainerDied","Data":"3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865"} Jul 18 02:32:42.068205 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:42.068118 2573 scope.go:117] "RemoveContainer" containerID="3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865" Jul 18 02:32:42.069173 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:42.069145 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:32:42.069256 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:42.069211 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:32:42.069256 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:42.069240 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:32:43.074214 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:43.074181 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" event={"ID":"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0","Type":"ContainerStarted","Data":"69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35"} Jul 18 02:32:43.225155 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:43.225078 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:43.325697 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:43.325613 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:44.005769 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:44.005727 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:44.069298 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:44.069258 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:44.070503 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:44.070478 2573 patch_prober.go:28] interesting pod/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 18 02:32:44.070632 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:44.070521 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 18 02:32:45.069435 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:45.069394 2573 patch_prober.go:28] interesting pod/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 18 02:32:45.069923 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:45.069456 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 18 02:32:46.069133 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:46.069083 2573 patch_prober.go:28] interesting pod/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 18 02:32:46.069321 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:46.069149 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 18 02:32:47.073245 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:47.073208 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:47.074439 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:47.074418 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:32:49.148019 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:49.147986 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:51.529821 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:51.529788 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:32:51.630625 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:32:51.630584 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:33:38.053811 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.053777 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq"] Jul 18 02:33:38.054263 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.054198 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee008ff9-50cb-4e45-82ff-7f90a75d1ab4" containerName="istio-proxy" Jul 18 02:33:38.055938 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.055922 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.058854 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.058831 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a45ea2\"/\"openshift-service-ca.crt\"" Jul 18 02:33:38.059807 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.059784 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a45ea2\"/\"test-model-a45ea2-kserve-self-signed-certs\"" Jul 18 02:33:38.059914 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.059825 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a45ea2\"/\"kube-root-ca.crt\"" Jul 18 02:33:38.060046 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.060033 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a45ea2\"/\"default-dockercfg-dldp9\"" Jul 18 02:33:38.071693 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.071666 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq"] Jul 18 02:33:38.077228 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.077205 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-tmp-dir\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.077334 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.077233 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g55w7\" (UniqueName: \"kubernetes.io/projected/2dd941d9-998a-492e-a44a-0c40ba62662d-kube-api-access-g55w7\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.077334 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.077251 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-dshm\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.077334 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.077278 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-home\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.077439 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.077333 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dd941d9-998a-492e-a44a-0c40ba62662d-tls-certs\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.077439 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.077362 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-model-cache\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.087393 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.087359 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:33:38.178455 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.178424 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-tmp-dir\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.178455 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.178458 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g55w7\" (UniqueName: \"kubernetes.io/projected/2dd941d9-998a-492e-a44a-0c40ba62662d-kube-api-access-g55w7\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.178713 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.178481 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-dshm\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.178713 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.178507 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-home\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.178713 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.178532 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dd941d9-998a-492e-a44a-0c40ba62662d-tls-certs\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.178713 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.178554 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-model-cache\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.178918 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.178866 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-tmp-dir\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.178918 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.178901 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-home\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.179080 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.179047 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-model-cache\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.180793 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.180772 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-dshm\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.181081 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.181061 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dd941d9-998a-492e-a44a-0c40ba62662d-tls-certs\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.192586 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.192566 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g55w7\" (UniqueName: \"kubernetes.io/projected/2dd941d9-998a-492e-a44a-0c40ba62662d-kube-api-access-g55w7\") pod \"test-model-a45ea2-kserve-5dd84b4745-b74jq\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.365325 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.365243 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:38.500281 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:38.500249 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq"] Jul 18 02:33:38.503287 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:33:38.503254 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dd941d9_998a_492e_a44a_0c40ba62662d.slice/crio-c6f891be0dfc8f385adddf767b7a7930341fa0f92bcafd0317b1b562b36fd60f WatchSource:0}: Error finding container c6f891be0dfc8f385adddf767b7a7930341fa0f92bcafd0317b1b562b36fd60f: Status 404 returned error can't find the container with id c6f891be0dfc8f385adddf767b7a7930341fa0f92bcafd0317b1b562b36fd60f Jul 18 02:33:39.357764 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:39.357728 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" event={"ID":"2dd941d9-998a-492e-a44a-0c40ba62662d","Type":"ContainerStarted","Data":"1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec"} Jul 18 02:33:39.357764 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:39.357773 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" event={"ID":"2dd941d9-998a-492e-a44a-0c40ba62662d","Type":"ContainerStarted","Data":"c6f891be0dfc8f385adddf767b7a7930341fa0f92bcafd0317b1b562b36fd60f"} Jul 18 02:33:39.383705 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:39.383663 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" podStartSLOduration=1.383648767 podStartE2EDuration="1.383648767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:33:39.381751382 +0000 UTC m=+2207.170087964" watchObservedRunningTime="2026-07-18 02:33:39.383648767 +0000 UTC m=+2207.171985349" Jul 18 02:33:48.365434 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:48.365391 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:48.365814 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:48.365451 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:48.377694 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:48.377671 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:48.414505 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:48.414480 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:33:48.631512 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:33:48.631482 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:03.464073 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:03.464040 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:04.433062 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:04.433020 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq"] Jul 18 02:34:04.457573 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:04.457543 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:05.102092 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:05.102048 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:05.202303 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:05.202269 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:15.017553 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:15.017521 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:19.452302 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.452262 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" podUID="2dd941d9-998a-492e-a44a-0c40ba62662d" containerName="main" containerID="cri-o://1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec" gracePeriod=45 Jul 18 02:34:19.636742 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.636713 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:19.690373 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.690344 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:34:19.818951 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.818890 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g55w7\" (UniqueName: \"kubernetes.io/projected/2dd941d9-998a-492e-a44a-0c40ba62662d-kube-api-access-g55w7\") pod \"2dd941d9-998a-492e-a44a-0c40ba62662d\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " Jul 18 02:34:19.819101 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.818956 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-dshm\") pod \"2dd941d9-998a-492e-a44a-0c40ba62662d\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " Jul 18 02:34:19.819101 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.819000 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-tmp-dir\") pod \"2dd941d9-998a-492e-a44a-0c40ba62662d\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " Jul 18 02:34:19.819101 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.819020 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-home\") pod \"2dd941d9-998a-492e-a44a-0c40ba62662d\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " Jul 18 02:34:19.819101 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.819071 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-model-cache\") pod \"2dd941d9-998a-492e-a44a-0c40ba62662d\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " Jul 18 02:34:19.819320 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.819103 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dd941d9-998a-492e-a44a-0c40ba62662d-tls-certs\") pod \"2dd941d9-998a-492e-a44a-0c40ba62662d\" (UID: \"2dd941d9-998a-492e-a44a-0c40ba62662d\") " Jul 18 02:34:19.819320 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.819212 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2dd941d9-998a-492e-a44a-0c40ba62662d" (UID: "2dd941d9-998a-492e-a44a-0c40ba62662d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:19.819320 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.819285 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-home" (OuterVolumeSpecName: "home") pod "2dd941d9-998a-492e-a44a-0c40ba62662d" (UID: "2dd941d9-998a-492e-a44a-0c40ba62662d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:19.819472 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.819415 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-model-cache" (OuterVolumeSpecName: "model-cache") pod "2dd941d9-998a-492e-a44a-0c40ba62662d" (UID: "2dd941d9-998a-492e-a44a-0c40ba62662d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:19.819472 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.819432 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-tmp-dir\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:19.819472 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.819454 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-home\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:19.821216 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.821192 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dd941d9-998a-492e-a44a-0c40ba62662d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2dd941d9-998a-492e-a44a-0c40ba62662d" (UID: "2dd941d9-998a-492e-a44a-0c40ba62662d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:34:19.821475 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.821454 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd941d9-998a-492e-a44a-0c40ba62662d-kube-api-access-g55w7" (OuterVolumeSpecName: "kube-api-access-g55w7") pod "2dd941d9-998a-492e-a44a-0c40ba62662d" (UID: "2dd941d9-998a-492e-a44a-0c40ba62662d"). InnerVolumeSpecName "kube-api-access-g55w7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:34:19.821547 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.821458 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-dshm" (OuterVolumeSpecName: "dshm") pod "2dd941d9-998a-492e-a44a-0c40ba62662d" (UID: "2dd941d9-998a-492e-a44a-0c40ba62662d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:19.920731 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.920697 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dd941d9-998a-492e-a44a-0c40ba62662d-tls-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:19.920731 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.920730 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g55w7\" (UniqueName: \"kubernetes.io/projected/2dd941d9-998a-492e-a44a-0c40ba62662d-kube-api-access-g55w7\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:19.920731 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.920741 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-dshm\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:19.920894 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:19.920750 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dd941d9-998a-492e-a44a-0c40ba62662d-model-cache\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:20.556216 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.556179 2573 generic.go:356] "Generic (PLEG): container finished" podID="2dd941d9-998a-492e-a44a-0c40ba62662d" containerID="1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec" exitCode=0 Jul 18 02:34:20.556216 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.556221 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" Jul 18 02:34:20.556688 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.556251 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" event={"ID":"2dd941d9-998a-492e-a44a-0c40ba62662d","Type":"ContainerDied","Data":"1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec"} Jul 18 02:34:20.556688 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.556285 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq" event={"ID":"2dd941d9-998a-492e-a44a-0c40ba62662d","Type":"ContainerDied","Data":"c6f891be0dfc8f385adddf767b7a7930341fa0f92bcafd0317b1b562b36fd60f"} Jul 18 02:34:20.556688 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.556306 2573 scope.go:117] "RemoveContainer" containerID="1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec" Jul 18 02:34:20.565385 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.565367 2573 scope.go:117] "RemoveContainer" containerID="1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec" Jul 18 02:34:20.565606 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:34:20.565589 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec\": container with ID starting with 1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec not found: ID does not exist" containerID="1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec" Jul 18 02:34:20.565653 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.565613 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec"} err="failed to get container status \"1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec\": rpc error: code = NotFound desc = could not find container \"1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec\": container with ID starting with 1e41c594d9baaeb18ea8b8e9ddfaaffa5ba8c5fb44dd4fc8f4c4d07bd89d9aec not found: ID does not exist" Jul 18 02:34:20.576569 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.576546 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq"] Jul 18 02:34:20.582560 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.582540 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a45ea2/test-model-a45ea2-kserve-5dd84b4745-b74jq"] Jul 18 02:34:20.722370 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.722339 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dd941d9-998a-492e-a44a-0c40ba62662d" path="/var/lib/kubelet/pods/2dd941d9-998a-492e-a44a-0c40ba62662d/volumes" Jul 18 02:34:20.893490 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:20.893458 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:25.799950 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:25.799902 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:25.858483 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:25.858454 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:26.059966 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:26.059882 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:26.284821 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:26.284776 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:26.920685 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:26.920653 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:27.018310 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:27.018277 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:36.952848 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:36.952806 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:41.474733 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:41.474703 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:41.842077 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:41.841954 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:47.912304 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:47.912264 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:47.983317 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:47.983284 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:48.151902 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:48.151870 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k"] Jul 18 02:34:48.152105 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:48.152084 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerName="istio-proxy" containerID="cri-o://69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35" gracePeriod=30 Jul 18 02:34:48.176841 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:48.176748 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:48.404796 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:48.404765 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:48.738546 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:48.738516 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:49.007460 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.007388 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr"] Jul 18 02:34:49.007788 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.007759 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd941d9-998a-492e-a44a-0c40ba62662d" containerName="main" Jul 18 02:34:49.007788 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.007771 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd941d9-998a-492e-a44a-0c40ba62662d" containerName="main" Jul 18 02:34:49.007898 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.007838 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dd941d9-998a-492e-a44a-0c40ba62662d" containerName="main" Jul 18 02:34:49.010862 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.010842 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.013133 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.013111 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6e5971-openshift-default-dockercfg-bnx8z\"" Jul 18 02:34:49.022090 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.022069 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr"] Jul 18 02:34:49.035560 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.035527 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:49.044826 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.044798 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99a24eae-60c6-436f-8162-e193dccec110-istiod-ca-cert\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.044943 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.044847 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-credential-socket\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.044943 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.044872 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-envoy\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.045121 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.045079 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-data\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.045190 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.045133 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-certs\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.045190 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.045171 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99a24eae-60c6-436f-8162-e193dccec110-istio-podinfo\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.045272 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.045200 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-istio-token\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.045307 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.045266 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-socket\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.045370 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.045353 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29p58\" (UniqueName: \"kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-kube-api-access-29p58\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.145874 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.145841 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29p58\" (UniqueName: \"kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-kube-api-access-29p58\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.145874 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.145878 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99a24eae-60c6-436f-8162-e193dccec110-istiod-ca-cert\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146120 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.145897 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-credential-socket\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146120 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.145912 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-envoy\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146120 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.145937 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-data\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146120 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.145964 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-certs\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146120 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.145999 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99a24eae-60c6-436f-8162-e193dccec110-istio-podinfo\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146120 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.146021 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-istio-token\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146120 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.146054 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-socket\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146499 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.146417 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-credential-socket\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146604 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.146577 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-data\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146668 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.146615 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-certs\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146725 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.146684 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-socket\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.146777 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.146724 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99a24eae-60c6-436f-8162-e193dccec110-istiod-ca-cert\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.148172 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.148145 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-envoy\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.148460 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.148440 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99a24eae-60c6-436f-8162-e193dccec110-istio-podinfo\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.153473 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.153447 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-istio-token\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.154254 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.154235 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29p58\" (UniqueName: \"kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-kube-api-access-29p58\") pod \"e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.320946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.320891 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:49.453799 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.453761 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr"] Jul 18 02:34:49.456544 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:34:49.456514 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a24eae_60c6_436f_8162_e193dccec110.slice/crio-b5322ae733ccceff1870627b97de1d108ce9ce25d8156a39875e403332b8b974 WatchSource:0}: Error finding container b5322ae733ccceff1870627b97de1d108ce9ce25d8156a39875e403332b8b974: Status 404 returned error can't find the container with id b5322ae733ccceff1870627b97de1d108ce9ce25d8156a39875e403332b8b974 Jul 18 02:34:49.458726 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.458694 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:34:49.458822 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.458758 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:34:49.458822 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.458789 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:34:49.624546 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.624510 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:49.706921 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.706868 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" event={"ID":"99a24eae-60c6-436f-8162-e193dccec110","Type":"ContainerStarted","Data":"6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2"} Jul 18 02:34:49.707112 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.706927 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" event={"ID":"99a24eae-60c6-436f-8162-e193dccec110","Type":"ContainerStarted","Data":"b5322ae733ccceff1870627b97de1d108ce9ce25d8156a39875e403332b8b974"} Jul 18 02:34:49.727019 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:49.726960 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" podStartSLOduration=1.726946206 podStartE2EDuration="1.726946206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:34:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:34:49.724915931 +0000 UTC m=+2277.513252534" watchObservedRunningTime="2026-07-18 02:34:49.726946206 +0000 UTC m=+2277.515282788" Jul 18 02:34:50.322008 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:50.321960 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:50.326662 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:50.326643 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:50.712327 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:50.712302 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:50.713421 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:50.713397 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:34:51.601832 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:51.601800 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:51.696663 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:51.696630 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:51.773428 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:51.773402 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:51.945266 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:51.945241 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:52.359146 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:52.359047 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:52.459579 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:52.459550 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:53.403578 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.403553 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:34:53.480655 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.480589 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-data\") pod \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " Jul 18 02:34:53.480655 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.480621 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-certs\") pod \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " Jul 18 02:34:53.480846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.480657 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istiod-ca-cert\") pod \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " Jul 18 02:34:53.480846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.480690 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thmxz\" (UniqueName: \"kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-kube-api-access-thmxz\") pod \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " Jul 18 02:34:53.480846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.480725 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-credential-socket\") pod \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " Jul 18 02:34:53.480846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.480793 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-token\") pod \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " Jul 18 02:34:53.480846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.480824 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-envoy\") pod \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " Jul 18 02:34:53.481142 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.480864 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-podinfo\") pod \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " Jul 18 02:34:53.481142 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.480866 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" (UID: "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:53.481142 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.480892 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-socket\") pod \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\" (UID: \"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0\") " Jul 18 02:34:53.481142 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.481107 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" (UID: "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:53.481142 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.481120 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" (UID: "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:34:53.481402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.481254 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:53.481402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.481273 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istiod-ca-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:53.481402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.481285 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-credential-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:53.481402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.481337 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" (UID: "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:53.481402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.481381 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" (UID: "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:53.482929 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.482902 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" (UID: "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:53.483107 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.482945 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-kube-api-access-thmxz" (OuterVolumeSpecName: "kube-api-access-thmxz") pod "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" (UID: "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0"). InnerVolumeSpecName "kube-api-access-thmxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:34:53.483180 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.483122 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" (UID: "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:34:53.483180 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.483141 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" (UID: "5f0efc41-4f6e-43a8-841c-b2a1a3af81c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:34:53.582172 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.582153 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-token\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:53.582172 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.582172 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-envoy\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:53.582305 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.582182 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-podinfo\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:53.582305 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.582189 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-workload-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:53.582305 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.582198 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-istio-data\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:53.582305 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.582208 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thmxz\" (UniqueName: \"kubernetes.io/projected/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0-kube-api-access-thmxz\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:34:53.729218 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.729183 2573 generic.go:356] "Generic (PLEG): container finished" podID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerID="69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35" exitCode=0 Jul 18 02:34:53.729391 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.729230 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" Jul 18 02:34:53.729391 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.729248 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" event={"ID":"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0","Type":"ContainerDied","Data":"69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35"} Jul 18 02:34:53.729391 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.729289 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k" event={"ID":"5f0efc41-4f6e-43a8-841c-b2a1a3af81c0","Type":"ContainerDied","Data":"c7b13c0fdbf78e27f877f756be5e437d2c5be1d5ad867ab1599d8561ccd0e375"} Jul 18 02:34:53.729391 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.729309 2573 scope.go:117] "RemoveContainer" containerID="69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35" Jul 18 02:34:53.739547 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.739528 2573 scope.go:117] "RemoveContainer" containerID="3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865" Jul 18 02:34:53.746920 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.746904 2573 scope.go:117] "RemoveContainer" containerID="69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35" Jul 18 02:34:53.747192 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:34:53.747172 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35\": container with ID starting with 69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35 not found: ID does not exist" containerID="69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35" Jul 18 02:34:53.747255 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.747200 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35"} err="failed to get container status \"69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35\": rpc error: code = NotFound desc = could not find container \"69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35\": container with ID starting with 69d1bdf842dac4973edbd014562bad32f40fbd051ea9f2da85000f7e10c99f35 not found: ID does not exist" Jul 18 02:34:53.747255 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.747218 2573 scope.go:117] "RemoveContainer" containerID="3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865" Jul 18 02:34:53.747437 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:34:53.747417 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865\": container with ID starting with 3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865 not found: ID does not exist" containerID="3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865" Jul 18 02:34:53.747483 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.747443 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865"} err="failed to get container status \"3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865\": rpc error: code = NotFound desc = could not find container \"3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865\": container with ID starting with 3bef8c0709a3e488d8a8280940bebebcf64a7a055ce11456f639d64d88c03865 not found: ID does not exist" Jul 18 02:34:53.752534 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.752511 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k"] Jul 18 02:34:53.758326 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:53.758306 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a45ea2-openshift-default-78f6484c5f-pbw7k"] Jul 18 02:34:54.722503 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:54.722471 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" path="/var/lib/kubelet/pods/5f0efc41-4f6e-43a8-841c-b2a1a3af81c0/volumes" Jul 18 02:34:56.086927 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:56.086864 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:56.539899 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:56.539851 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:56.831332 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:56.831240 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:56.930422 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:56.930391 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:59.519988 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:59.519903 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:59.600508 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:59.600468 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:34:59.675036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:34:59.674994 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:00.141079 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:00.141042 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:00.373596 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:00.373562 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:00.481017 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:00.480928 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:03.965852 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:03.965813 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:06.754158 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:06.754110 2573 patch_prober.go:28] interesting pod/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 18 02:35:06.754542 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:06.754174 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 18 02:35:07.812181 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:07.812150 2573 generic.go:356] "Generic (PLEG): container finished" podID="99a24eae-60c6-436f-8162-e193dccec110" containerID="6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2" exitCode=0 Jul 18 02:35:07.812641 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:07.812210 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" event={"ID":"99a24eae-60c6-436f-8162-e193dccec110","Type":"ContainerDied","Data":"6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2"} Jul 18 02:35:07.812641 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:07.812512 2573 scope.go:117] "RemoveContainer" containerID="6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2" Jul 18 02:35:07.813446 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:07.813414 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:35:07.813510 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:07.813481 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:35:07.813546 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:07.813515 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:35:08.818871 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:08.818839 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" event={"ID":"99a24eae-60c6-436f-8162-e193dccec110","Type":"ContainerStarted","Data":"528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b"} Jul 18 02:35:09.321750 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:09.321718 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:35:09.321938 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:09.321911 2573 patch_prober.go:28] interesting pod/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 18 02:35:09.322013 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:09.321954 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 18 02:35:10.321367 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:10.321328 2573 patch_prober.go:28] interesting pod/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 18 02:35:10.321767 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:10.321403 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 18 02:35:11.321738 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:11.321695 2573 patch_prober.go:28] interesting pod/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 18 02:35:11.322177 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:11.321772 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 18 02:35:12.321360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:12.321321 2573 patch_prober.go:28] interesting pod/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 18 02:35:12.321545 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:12.321392 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 18 02:35:13.244337 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:13.244301 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:13.324648 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:13.324623 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:35:13.324925 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:13.324904 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:35:13.325617 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:13.325598 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:35:13.351528 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:13.351495 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:22.137003 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:22.136938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:22.238582 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:22.238550 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:27.532770 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:27.532739 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:27.634371 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:27.634340 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:32.801795 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:32.801756 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:37.073140 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:37.073089 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:37.457686 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:37.457659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:43.031721 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:43.031686 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:43.133353 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:43.133309 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:43.274132 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:43.274102 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr"] Jul 18 02:35:43.274328 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:43.274305 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" containerID="cri-o://528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b" gracePeriod=30 Jul 18 02:35:43.293953 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:43.293871 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:43.327471 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:43.327441 2573 patch_prober.go:28] interesting pod/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 02:35:43.327606 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:43.327499 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 02:35:43.520176 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:43.520141 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:44.301262 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:44.301230 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:44.402277 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:44.402247 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:35:48.528093 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.528070 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:35:48.638190 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638165 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-certs\") pod \"99a24eae-60c6-436f-8162-e193dccec110\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " Jul 18 02:35:48.638368 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638200 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-socket\") pod \"99a24eae-60c6-436f-8162-e193dccec110\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " Jul 18 02:35:48.638368 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638222 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29p58\" (UniqueName: \"kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-kube-api-access-29p58\") pod \"99a24eae-60c6-436f-8162-e193dccec110\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " Jul 18 02:35:48.638368 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638253 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-data\") pod \"99a24eae-60c6-436f-8162-e193dccec110\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " Jul 18 02:35:48.638368 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638278 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-envoy\") pod \"99a24eae-60c6-436f-8162-e193dccec110\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " Jul 18 02:35:48.638368 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638302 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-credential-socket\") pod \"99a24eae-60c6-436f-8162-e193dccec110\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " Jul 18 02:35:48.638368 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638323 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99a24eae-60c6-436f-8162-e193dccec110-istio-podinfo\") pod \"99a24eae-60c6-436f-8162-e193dccec110\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " Jul 18 02:35:48.638368 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638354 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-istio-token\") pod \"99a24eae-60c6-436f-8162-e193dccec110\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " Jul 18 02:35:48.638744 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638384 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99a24eae-60c6-436f-8162-e193dccec110-istiod-ca-cert\") pod \"99a24eae-60c6-436f-8162-e193dccec110\" (UID: \"99a24eae-60c6-436f-8162-e193dccec110\") " Jul 18 02:35:48.638744 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638495 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99a24eae-60c6-436f-8162-e193dccec110" (UID: "99a24eae-60c6-436f-8162-e193dccec110"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:35:48.638744 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638574 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99a24eae-60c6-436f-8162-e193dccec110" (UID: "99a24eae-60c6-436f-8162-e193dccec110"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:35:48.638744 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638661 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:35:48.638744 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638705 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99a24eae-60c6-436f-8162-e193dccec110" (UID: "99a24eae-60c6-436f-8162-e193dccec110"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:35:48.639000 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.638799 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a24eae-60c6-436f-8162-e193dccec110-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99a24eae-60c6-436f-8162-e193dccec110" (UID: "99a24eae-60c6-436f-8162-e193dccec110"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:35:48.639480 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.639452 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-data" (OuterVolumeSpecName: "istio-data") pod "99a24eae-60c6-436f-8162-e193dccec110" (UID: "99a24eae-60c6-436f-8162-e193dccec110"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:35:48.640499 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.640469 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99a24eae-60c6-436f-8162-e193dccec110-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99a24eae-60c6-436f-8162-e193dccec110" (UID: "99a24eae-60c6-436f-8162-e193dccec110"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:35:48.640604 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.640572 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-istio-token" (OuterVolumeSpecName: "istio-token") pod "99a24eae-60c6-436f-8162-e193dccec110" (UID: "99a24eae-60c6-436f-8162-e193dccec110"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:35:48.640764 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.640746 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-kube-api-access-29p58" (OuterVolumeSpecName: "kube-api-access-29p58") pod "99a24eae-60c6-436f-8162-e193dccec110" (UID: "99a24eae-60c6-436f-8162-e193dccec110"). InnerVolumeSpecName "kube-api-access-29p58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:35:48.640820 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.640761 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99a24eae-60c6-436f-8162-e193dccec110" (UID: "99a24eae-60c6-436f-8162-e193dccec110"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:35:48.739438 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.739413 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-workload-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:35:48.739438 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.739438 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29p58\" (UniqueName: \"kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-kube-api-access-29p58\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:35:48.739590 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.739452 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-data\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:35:48.739590 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.739464 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-istio-envoy\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:35:48.739590 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.739475 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a24eae-60c6-436f-8162-e193dccec110-credential-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:35:48.739590 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.739486 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99a24eae-60c6-436f-8162-e193dccec110-istio-podinfo\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:35:48.739590 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.739498 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99a24eae-60c6-436f-8162-e193dccec110-istio-token\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:35:48.739590 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:48.739510 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99a24eae-60c6-436f-8162-e193dccec110-istiod-ca-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:35:49.005013 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.004904 2573 generic.go:356] "Generic (PLEG): container finished" podID="99a24eae-60c6-436f-8162-e193dccec110" containerID="528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b" exitCode=0 Jul 18 02:35:49.005013 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.004957 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" Jul 18 02:35:49.005013 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.004962 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" event={"ID":"99a24eae-60c6-436f-8162-e193dccec110","Type":"ContainerDied","Data":"528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b"} Jul 18 02:35:49.005300 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.005022 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr" event={"ID":"99a24eae-60c6-436f-8162-e193dccec110","Type":"ContainerDied","Data":"b5322ae733ccceff1870627b97de1d108ce9ce25d8156a39875e403332b8b974"} Jul 18 02:35:49.005300 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.005042 2573 scope.go:117] "RemoveContainer" containerID="528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b" Jul 18 02:35:49.015789 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.015773 2573 scope.go:117] "RemoveContainer" containerID="6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2" Jul 18 02:35:49.024773 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.024755 2573 scope.go:117] "RemoveContainer" containerID="528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b" Jul 18 02:35:49.025048 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:35:49.025025 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b\": container with ID starting with 528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b not found: ID does not exist" containerID="528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b" Jul 18 02:35:49.025145 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.025060 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b"} err="failed to get container status \"528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b\": rpc error: code = NotFound desc = could not find container \"528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b\": container with ID starting with 528b40dad269e9d2adfcb9c507bc7fab85854b2a68bf07ea220d44167706aa2b not found: ID does not exist" Jul 18 02:35:49.025145 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.025083 2573 scope.go:117] "RemoveContainer" containerID="6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2" Jul 18 02:35:49.025354 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:35:49.025327 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2\": container with ID starting with 6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2 not found: ID does not exist" containerID="6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2" Jul 18 02:35:49.025421 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.025364 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2"} err="failed to get container status \"6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2\": rpc error: code = NotFound desc = could not find container \"6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2\": container with ID starting with 6b58bdbd98eb0a38b622c4bd777166fc5f9355985df8752a89d24a8091a189b2 not found: ID does not exist" Jul 18 02:35:49.025707 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.025685 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr"] Jul 18 02:35:49.028002 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:49.027982 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6e5971-openshift-default-dbdffcbfc-fkdtr"] Jul 18 02:35:50.728804 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:50.728767 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99a24eae-60c6-436f-8162-e193dccec110" path="/var/lib/kubelet/pods/99a24eae-60c6-436f-8162-e193dccec110/volumes" Jul 18 02:35:59.105814 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:35:59.105781 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:03.726388 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:03.726353 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:04.119709 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:04.119627 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:09.755739 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:09.755698 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:09.817721 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:09.817688 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:10.024318 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.024232 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:10.244213 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.244176 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:10.561492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.561456 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:10.838833 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.838760 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs"] Jul 18 02:36:10.839225 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.839209 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" Jul 18 02:36:10.839265 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.839228 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" Jul 18 02:36:10.839265 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.839239 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" Jul 18 02:36:10.839265 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.839248 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" Jul 18 02:36:10.839355 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.839264 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerName="istio-proxy" Jul 18 02:36:10.839355 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.839272 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerName="istio-proxy" Jul 18 02:36:10.839421 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.839380 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerName="istio-proxy" Jul 18 02:36:10.839421 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.839390 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" Jul 18 02:36:10.839421 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.839398 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerName="istio-proxy" Jul 18 02:36:10.839421 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.839408 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="99a24eae-60c6-436f-8162-e193dccec110" containerName="istio-proxy" Jul 18 02:36:10.842529 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.842511 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:10.844997 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.844959 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-0d6b43-openshift-default-dockercfg-rdj6d\"" Jul 18 02:36:10.852406 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.852383 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs"] Jul 18 02:36:10.861428 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.861402 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:10.925523 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.925492 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-data\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:10.925669 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.925526 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt9g5\" (UniqueName: \"kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-kube-api-access-rt9g5\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:10.925669 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.925545 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-envoy\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:10.925669 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.925593 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-token\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:10.925669 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.925636 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-podinfo\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:10.925817 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.925683 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-credential-socket\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:10.925817 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.925712 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-certs\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:10.925817 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.925727 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621de3d6-205d-44cb-a382-bd2a61c82d50-istiod-ca-cert\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:10.925817 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.925791 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-socket\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:10.962629 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:10.962599 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:11.026754 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.026719 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-credential-socket\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.026946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.026761 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-certs\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.026946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.026788 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621de3d6-205d-44cb-a382-bd2a61c82d50-istiod-ca-cert\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.026946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.026838 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-socket\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.026946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.026880 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-data\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.026946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.026908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt9g5\" (UniqueName: \"kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-kube-api-access-rt9g5\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.026946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.026932 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-envoy\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.027278 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.026960 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-token\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.027278 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.027020 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-podinfo\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.027278 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.027220 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-certs\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.027278 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.027254 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-credential-socket\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.027483 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.027363 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-socket\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.027483 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.027456 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-data\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.027650 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.027634 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621de3d6-205d-44cb-a382-bd2a61c82d50-istiod-ca-cert\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.029403 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.029374 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-envoy\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.029526 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.029489 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-podinfo\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.035939 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.035915 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-token\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.036235 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.036204 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt9g5\" (UniqueName: \"kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-kube-api-access-rt9g5\") pod \"e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.153590 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.153561 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:11.284479 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.284455 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs"] Jul 18 02:36:11.286123 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:36:11.286098 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621de3d6_205d_44cb_a382_bd2a61c82d50.slice/crio-b9eeb666c147dbed7048de71f495c63846e636f17ebfb1dd0485dbce520d94ec WatchSource:0}: Error finding container b9eeb666c147dbed7048de71f495c63846e636f17ebfb1dd0485dbce520d94ec: Status 404 returned error can't find the container with id b9eeb666c147dbed7048de71f495c63846e636f17ebfb1dd0485dbce520d94ec Jul 18 02:36:11.288480 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.288443 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:36:11.288574 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.288526 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:36:11.288611 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:11.288568 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:36:12.112941 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:12.112909 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" event={"ID":"621de3d6-205d-44cb-a382-bd2a61c82d50","Type":"ContainerStarted","Data":"0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23"} Jul 18 02:36:12.113340 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:12.112956 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" event={"ID":"621de3d6-205d-44cb-a382-bd2a61c82d50","Type":"ContainerStarted","Data":"b9eeb666c147dbed7048de71f495c63846e636f17ebfb1dd0485dbce520d94ec"} Jul 18 02:36:12.132779 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:12.132737 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" podStartSLOduration=2.132722791 podStartE2EDuration="2.132722791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:36:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:36:12.1312729 +0000 UTC m=+2359.919609507" watchObservedRunningTime="2026-07-18 02:36:12.132722791 +0000 UTC m=+2359.921059373" Jul 18 02:36:12.154129 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:12.154107 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:12.158843 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:12.158824 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:13.117779 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:13.117746 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:13.118688 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:13.118671 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:14.014582 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:14.014555 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:14.117577 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:14.117531 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:14.213513 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:14.213474 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:14.513821 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:14.513788 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:14.751611 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:14.751578 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:14.841783 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:14.841694 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:18.507193 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:18.507161 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:18.836895 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:18.836789 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:19.144209 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:19.144171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:19.362765 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:19.362736 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:21.960825 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:21.960778 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:22.050457 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:22.050418 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:22.115315 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:22.115282 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:22.357960 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:22.357922 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:22.701634 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:22.701594 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:22.801615 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:22.801577 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:26.433310 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:26.433281 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:28.119741 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:28.119704 2573 patch_prober.go:28] interesting pod/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 18 02:36:28.120306 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:28.119778 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 18 02:36:28.206262 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:28.206229 2573 generic.go:356] "Generic (PLEG): container finished" podID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerID="0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23" exitCode=0 Jul 18 02:36:28.206424 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:28.206306 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" event={"ID":"621de3d6-205d-44cb-a382-bd2a61c82d50","Type":"ContainerDied","Data":"0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23"} Jul 18 02:36:28.206779 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:28.206760 2573 scope.go:117] "RemoveContainer" containerID="0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23" Jul 18 02:36:28.207755 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:28.207726 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:36:28.207823 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:28.207806 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:36:28.207871 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:28.207858 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 18 02:36:29.213409 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:29.213376 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" event={"ID":"621de3d6-205d-44cb-a382-bd2a61c82d50","Type":"ContainerStarted","Data":"b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63"} Jul 18 02:36:30.154682 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:30.154652 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:30.154988 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:30.154945 2573 patch_prober.go:28] interesting pod/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 18 02:36:30.155112 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:30.155012 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 18 02:36:31.154927 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:31.154888 2573 patch_prober.go:28] interesting pod/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 18 02:36:31.155340 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:31.154954 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 18 02:36:32.154400 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:32.154370 2573 patch_prober.go:28] interesting pod/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 18 02:36:32.154576 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:32.154419 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 18 02:36:33.158234 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:33.158201 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:33.158697 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:33.158461 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:33.159085 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:33.159071 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:36:33.647985 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:33.647950 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:36.588863 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:36.588830 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:36.689404 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:36.689379 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:42.450428 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:42.450389 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:44.663194 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:44.663157 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:44.764238 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:44.764197 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:49.280984 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:49.280942 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:52.822925 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:52.822889 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:36:52.830104 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:52.830083 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:36:55.326373 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:55.326342 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:36:55.426679 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:36:55.426647 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:37:08.629434 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.629402 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx"] Jul 18 02:37:08.630032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.629796 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerName="istio-proxy" Jul 18 02:37:08.630032 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.629812 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0efc41-4f6e-43a8-841c-b2a1a3af81c0" containerName="istio-proxy" Jul 18 02:37:08.632347 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.632327 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.635802 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.635780 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"kube-root-ca.crt\"" Jul 18 02:37:08.635895 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.635847 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"openshift-service-ca.crt\"" Jul 18 02:37:08.636030 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.636003 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"auth-test-model-922cd9-kserve-self-signed-certs\"" Jul 18 02:37:08.636134 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.636120 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"default-dockercfg-xwlkk\"" Jul 18 02:37:08.643126 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.643109 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx"] Jul 18 02:37:08.670670 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.670650 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:37:08.691919 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.691897 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-dshm\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.692025 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.691931 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-home\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.692025 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.691946 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tmp-dir\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.692025 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.692020 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29z5h\" (UniqueName: \"kubernetes.io/projected/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-kube-api-access-29z5h\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.692155 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.692038 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tls-certs\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.692155 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.692061 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-model-cache\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.792734 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.792707 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tmp-dir\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.792836 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.792756 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29z5h\" (UniqueName: \"kubernetes.io/projected/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-kube-api-access-29z5h\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.792883 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.792865 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tls-certs\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.792940 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.792920 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-model-cache\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.793039 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.793022 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-dshm\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.793102 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.793075 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-home\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.793169 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.793149 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tmp-dir\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.793283 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.793261 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-model-cache\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.793407 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.793391 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-home\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.795152 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.795129 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-dshm\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.795316 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.795299 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tls-certs\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.800956 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.800933 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29z5h\" (UniqueName: \"kubernetes.io/projected/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-kube-api-access-29z5h\") pod \"auth-test-model-922cd9-kserve-7b869c4759-5frlx\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:08.942872 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:08.942791 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:09.065552 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:09.065523 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx"] Jul 18 02:37:09.067046 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:37:09.067023 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4f16aef_3e90_4a20_964f_8f5ffbfc3e33.slice/crio-2dda6045893a584fa4d019668fe8d8ddafa27a704eabe05b4ae86ea9ab2d49a1 WatchSource:0}: Error finding container 2dda6045893a584fa4d019668fe8d8ddafa27a704eabe05b4ae86ea9ab2d49a1: Status 404 returned error can't find the container with id 2dda6045893a584fa4d019668fe8d8ddafa27a704eabe05b4ae86ea9ab2d49a1 Jul 18 02:37:09.408211 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:09.408174 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" event={"ID":"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33","Type":"ContainerStarted","Data":"50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457"} Jul 18 02:37:09.408359 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:09.408218 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" event={"ID":"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33","Type":"ContainerStarted","Data":"2dda6045893a584fa4d019668fe8d8ddafa27a704eabe05b4ae86ea9ab2d49a1"} Jul 18 02:37:09.425052 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:09.425008 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" podStartSLOduration=1.4249682209999999 podStartE2EDuration="1.424968221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:37:09.424182152 +0000 UTC m=+2417.212518732" watchObservedRunningTime="2026-07-18 02:37:09.424968221 +0000 UTC m=+2417.213304802" Jul 18 02:37:18.943236 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:18.943150 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:18.943665 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:18.943413 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:18.955383 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:18.955357 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:37:19.465851 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:37:19.465821 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:39:16.444000 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:16.443897 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:39:17.312790 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:17.312754 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx"] Jul 18 02:39:17.331846 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:17.331816 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:39:31.761426 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:31.761394 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:39:32.334207 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.334136 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" podUID="a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" containerName="main" containerID="cri-o://50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457" gracePeriod=45 Jul 18 02:39:32.587982 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.587908 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:39:32.676774 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.676749 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tmp-dir\") pod \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " Jul 18 02:39:32.676959 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.676786 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-dshm\") pod \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " Jul 18 02:39:32.676959 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.676802 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29z5h\" (UniqueName: \"kubernetes.io/projected/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-kube-api-access-29z5h\") pod \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " Jul 18 02:39:32.676959 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.676826 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-home\") pod \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " Jul 18 02:39:32.676959 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.676852 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tls-certs\") pod \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " Jul 18 02:39:32.676959 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.676900 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-model-cache\") pod \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\" (UID: \"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33\") " Jul 18 02:39:32.677260 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.677095 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" (UID: "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:39:32.677260 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.677109 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-home" (OuterVolumeSpecName: "home") pod "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" (UID: "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:39:32.677260 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.677217 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-home\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:39:32.677260 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.677232 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tmp-dir\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:39:32.677401 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.677257 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" (UID: "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:39:32.679047 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.679025 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" (UID: "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:39:32.679148 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.679129 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-kube-api-access-29z5h" (OuterVolumeSpecName: "kube-api-access-29z5h") pod "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" (UID: "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33"). InnerVolumeSpecName "kube-api-access-29z5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:39:32.679148 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.679140 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-dshm" (OuterVolumeSpecName: "dshm") pod "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" (UID: "a4f16aef-3e90-4a20-964f-8f5ffbfc3e33"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:39:32.778062 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.778036 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-model-cache\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:39:32.778062 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.778059 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-dshm\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:39:32.778062 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.778068 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29z5h\" (UniqueName: \"kubernetes.io/projected/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-kube-api-access-29z5h\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:39:32.778443 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:32.778076 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33-tls-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:39:33.089477 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:33.089441 2573 generic.go:356] "Generic (PLEG): container finished" podID="a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" containerID="50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457" exitCode=0 Jul 18 02:39:33.089643 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:33.089488 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" Jul 18 02:39:33.089643 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:33.089524 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" event={"ID":"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33","Type":"ContainerDied","Data":"50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457"} Jul 18 02:39:33.089643 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:33.089565 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx" event={"ID":"a4f16aef-3e90-4a20-964f-8f5ffbfc3e33","Type":"ContainerDied","Data":"2dda6045893a584fa4d019668fe8d8ddafa27a704eabe05b4ae86ea9ab2d49a1"} Jul 18 02:39:33.089643 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:33.089581 2573 scope.go:117] "RemoveContainer" containerID="50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457" Jul 18 02:39:33.098107 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:33.098086 2573 scope.go:117] "RemoveContainer" containerID="50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457" Jul 18 02:39:33.098370 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:39:33.098349 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457\": container with ID starting with 50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457 not found: ID does not exist" containerID="50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457" Jul 18 02:39:33.098434 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:33.098386 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457"} err="failed to get container status \"50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457\": rpc error: code = NotFound desc = could not find container \"50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457\": container with ID starting with 50852382f811816324ef1891334d55b2ea140a9afdd08a59af3c11ce1217d457 not found: ID does not exist" Jul 18 02:39:33.106264 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:33.106242 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx"] Jul 18 02:39:33.110213 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:33.110192 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/auth-test-model-922cd9-kserve-7b869c4759-5frlx"] Jul 18 02:39:34.721871 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:39:34.721828 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" path="/var/lib/kubelet/pods/a4f16aef-3e90-4a20-964f-8f5ffbfc3e33/volumes" Jul 18 02:40:07.043499 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:07.043466 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:40:13.522378 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:13.522338 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:40:16.839910 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.839836 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6"] Jul 18 02:40:16.840260 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.840209 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" containerName="main" Jul 18 02:40:16.840260 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.840224 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" containerName="main" Jul 18 02:40:16.840331 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.840285 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4f16aef-3e90-4a20-964f-8f5ffbfc3e33" containerName="main" Jul 18 02:40:16.843346 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.843327 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:16.846505 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.846484 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"kube-root-ca.crt\"" Jul 18 02:40:16.846505 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.846493 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"sub-test-model-922cd9-kserve-self-signed-certs\"" Jul 18 02:40:16.846699 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.846487 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"default-dockercfg-xwlkk\"" Jul 18 02:40:16.846699 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.846493 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"openshift-service-ca.crt\"" Jul 18 02:40:16.854092 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.854070 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6"] Jul 18 02:40:16.869583 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.869555 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:40:16.950214 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.950178 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-dshm\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:16.950214 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.950215 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dv7d\" (UniqueName: \"kubernetes.io/projected/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-kube-api-access-9dv7d\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:16.950456 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.950295 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-home\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:16.950456 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.950313 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tls-certs\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:16.950456 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.950381 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-model-cache\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:16.950591 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:16.950458 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tmp-dir\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.051031 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.050963 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tmp-dir\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.051031 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.051040 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-dshm\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.051286 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.051057 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dv7d\" (UniqueName: \"kubernetes.io/projected/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-kube-api-access-9dv7d\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.051286 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.051116 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-home\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.051286 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.051131 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tls-certs\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.051286 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.051160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-model-cache\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.051489 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.051408 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tmp-dir\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.051489 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.051431 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-home\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.051489 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.051473 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-model-cache\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.053141 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.053119 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-dshm\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.053564 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.053547 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tls-certs\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.061584 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.061565 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dv7d\" (UniqueName: \"kubernetes.io/projected/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-kube-api-access-9dv7d\") pod \"sub-test-model-922cd9-kserve-749b7dd96f-wqmb6\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.152231 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.152203 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:17.276710 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.276689 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6"] Jul 18 02:40:17.278926 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:40:17.278898 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb39b3c2a_9ada_4713_8c8f_f9318afa33ed.slice/crio-9b96378a3fa452e7e24ed8c2d328633c6b343c0bbfc935878cb9740b0b400722 WatchSource:0}: Error finding container 9b96378a3fa452e7e24ed8c2d328633c6b343c0bbfc935878cb9740b0b400722: Status 404 returned error can't find the container with id 9b96378a3fa452e7e24ed8c2d328633c6b343c0bbfc935878cb9740b0b400722 Jul 18 02:40:17.301949 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:17.301925 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" event={"ID":"b39b3c2a-9ada-4713-8c8f-f9318afa33ed","Type":"ContainerStarted","Data":"9b96378a3fa452e7e24ed8c2d328633c6b343c0bbfc935878cb9740b0b400722"} Jul 18 02:40:18.307831 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:18.307795 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" event={"ID":"b39b3c2a-9ada-4713-8c8f-f9318afa33ed","Type":"ContainerStarted","Data":"721c06296a8474f81428eaea3f7be17a468063cf8c19d1b308caaf0d252c27b1"} Jul 18 02:40:18.331297 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:18.331252 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" podStartSLOduration=2.331234669 podStartE2EDuration="2.331234669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:18.32981331 +0000 UTC m=+2606.118149892" watchObservedRunningTime="2026-07-18 02:40:18.331234669 +0000 UTC m=+2606.119571251" Jul 18 02:40:27.152579 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:27.152545 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:27.152998 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:27.152604 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:27.164632 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:27.164611 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:40:27.359174 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:40:27.359149 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:41:02.171561 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:02.171527 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:41:22.475460 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:22.475421 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:41:23.438289 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:23.438256 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6"] Jul 18 02:41:23.460536 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:23.460506 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:41:31.062040 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:31.062003 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:41:37.365812 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:37.365775 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:41:37.467132 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:37.467100 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:41:38.457563 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.457527 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" podUID="b39b3c2a-9ada-4713-8c8f-f9318afa33ed" containerName="main" containerID="cri-o://721c06296a8474f81428eaea3f7be17a468063cf8c19d1b308caaf0d252c27b1" gracePeriod=45 Jul 18 02:41:38.689151 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.689127 2573 generic.go:356] "Generic (PLEG): container finished" podID="b39b3c2a-9ada-4713-8c8f-f9318afa33ed" containerID="721c06296a8474f81428eaea3f7be17a468063cf8c19d1b308caaf0d252c27b1" exitCode=0 Jul 18 02:41:38.689264 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.689198 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" event={"ID":"b39b3c2a-9ada-4713-8c8f-f9318afa33ed","Type":"ContainerDied","Data":"721c06296a8474f81428eaea3f7be17a468063cf8c19d1b308caaf0d252c27b1"} Jul 18 02:41:38.689264 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.689242 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" event={"ID":"b39b3c2a-9ada-4713-8c8f-f9318afa33ed","Type":"ContainerDied","Data":"9b96378a3fa452e7e24ed8c2d328633c6b343c0bbfc935878cb9740b0b400722"} Jul 18 02:41:38.689264 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.689259 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b96378a3fa452e7e24ed8c2d328633c6b343c0bbfc935878cb9740b0b400722" Jul 18 02:41:38.692636 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.692620 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:41:38.803838 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.803771 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-model-cache\") pod \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " Jul 18 02:41:38.803838 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.803813 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-home\") pod \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " Jul 18 02:41:38.803838 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.803831 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tls-certs\") pod \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " Jul 18 02:41:38.804126 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.803853 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tmp-dir\") pod \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " Jul 18 02:41:38.804126 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.803900 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-dshm\") pod \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " Jul 18 02:41:38.804126 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.803959 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dv7d\" (UniqueName: \"kubernetes.io/projected/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-kube-api-access-9dv7d\") pod \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\" (UID: \"b39b3c2a-9ada-4713-8c8f-f9318afa33ed\") " Jul 18 02:41:38.804126 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.804067 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "b39b3c2a-9ada-4713-8c8f-f9318afa33ed" (UID: "b39b3c2a-9ada-4713-8c8f-f9318afa33ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:41:38.804126 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.804085 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-home" (OuterVolumeSpecName: "home") pod "b39b3c2a-9ada-4713-8c8f-f9318afa33ed" (UID: "b39b3c2a-9ada-4713-8c8f-f9318afa33ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:41:38.804334 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.804165 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b39b3c2a-9ada-4713-8c8f-f9318afa33ed" (UID: "b39b3c2a-9ada-4713-8c8f-f9318afa33ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:41:38.804334 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.804250 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-model-cache\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:41:38.804334 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.804262 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-home\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:41:38.804334 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.804270 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tmp-dir\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:41:38.805898 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.805878 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b39b3c2a-9ada-4713-8c8f-f9318afa33ed" (UID: "b39b3c2a-9ada-4713-8c8f-f9318afa33ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:41:38.805999 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.805962 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-dshm" (OuterVolumeSpecName: "dshm") pod "b39b3c2a-9ada-4713-8c8f-f9318afa33ed" (UID: "b39b3c2a-9ada-4713-8c8f-f9318afa33ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:41:38.806055 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.805995 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-kube-api-access-9dv7d" (OuterVolumeSpecName: "kube-api-access-9dv7d") pod "b39b3c2a-9ada-4713-8c8f-f9318afa33ed" (UID: "b39b3c2a-9ada-4713-8c8f-f9318afa33ed"). InnerVolumeSpecName "kube-api-access-9dv7d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:41:38.904782 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.904759 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dv7d\" (UniqueName: \"kubernetes.io/projected/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-kube-api-access-9dv7d\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:41:38.904782 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.904780 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-tls-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:41:38.904936 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:38.904793 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39b3c2a-9ada-4713-8c8f-f9318afa33ed-dshm\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:41:39.693950 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:39.693918 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6" Jul 18 02:41:39.718155 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:39.718131 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6"] Jul 18 02:41:39.720054 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:39.720026 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/sub-test-model-922cd9-kserve-749b7dd96f-wqmb6"] Jul 18 02:41:40.722375 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:40.722339 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b39b3c2a-9ada-4713-8c8f-f9318afa33ed" path="/var/lib/kubelet/pods/b39b3c2a-9ada-4713-8c8f-f9318afa33ed/volumes" Jul 18 02:41:45.581201 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:45.581108 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:41:45.682476 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:45.682442 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:41:52.632494 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.632460 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th"] Jul 18 02:41:52.632848 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.632773 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b39b3c2a-9ada-4713-8c8f-f9318afa33ed" containerName="main" Jul 18 02:41:52.632848 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.632784 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b39b3c2a-9ada-4713-8c8f-f9318afa33ed" containerName="main" Jul 18 02:41:52.632848 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.632847 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b39b3c2a-9ada-4713-8c8f-f9318afa33ed" containerName="main" Jul 18 02:41:52.635862 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.635845 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.638166 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.638143 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"openshift-service-ca.crt\"" Jul 18 02:41:52.638301 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.638185 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"default-dockercfg-xwlkk\"" Jul 18 02:41:52.639031 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.639016 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"rate-test-model-922cd9-kserve-self-signed-certs\"" Jul 18 02:41:52.639103 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.639034 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-922cd9\"/\"kube-root-ca.crt\"" Jul 18 02:41:52.644000 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.643964 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th"] Jul 18 02:41:52.654687 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.654659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:41:52.714217 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.714187 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85840a40-e5f9-49a6-9356-55f239d63b4f-tls-certs\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.714357 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.714231 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-dshm\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.714357 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.714253 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-tmp-dir\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.714432 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.714352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-home\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.714432 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.714378 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-model-cache\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.714496 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.714443 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27gm5\" (UniqueName: \"kubernetes.io/projected/85840a40-e5f9-49a6-9356-55f239d63b4f-kube-api-access-27gm5\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.815070 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.815032 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-home\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.815070 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.815069 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-model-cache\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.815325 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.815111 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27gm5\" (UniqueName: \"kubernetes.io/projected/85840a40-e5f9-49a6-9356-55f239d63b4f-kube-api-access-27gm5\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.815325 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.815132 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85840a40-e5f9-49a6-9356-55f239d63b4f-tls-certs\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.815325 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.815163 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-dshm\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.815325 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.815179 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-tmp-dir\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.815535 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.815437 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-home\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.815535 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.815513 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-model-cache\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.815611 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.815543 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-tmp-dir\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.817426 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.817402 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-dshm\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.817672 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.817519 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85840a40-e5f9-49a6-9356-55f239d63b4f-tls-certs\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.823067 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.823049 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27gm5\" (UniqueName: \"kubernetes.io/projected/85840a40-e5f9-49a6-9356-55f239d63b4f-kube-api-access-27gm5\") pod \"rate-test-model-922cd9-kserve-6875699f-lx8th\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:52.854777 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.854754 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:41:52.861820 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.861803 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dgqnb_2e4eb8b4-5a82-4673-a734-1a2d3842c1a8/ovn-acl-logging/0.log" Jul 18 02:41:52.945492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:52.945403 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:41:53.074556 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:53.074533 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th"] Jul 18 02:41:53.075851 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:41:53.075829 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85840a40_e5f9_49a6_9356_55f239d63b4f.slice/crio-ed267982a5ceebf89205045f6e043a11d92fa9f95df7c4982b7db1643d587260 WatchSource:0}: Error finding container ed267982a5ceebf89205045f6e043a11d92fa9f95df7c4982b7db1643d587260: Status 404 returned error can't find the container with id ed267982a5ceebf89205045f6e043a11d92fa9f95df7c4982b7db1643d587260 Jul 18 02:41:53.762218 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:53.762187 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" event={"ID":"85840a40-e5f9-49a6-9356-55f239d63b4f","Type":"ContainerStarted","Data":"7782d4e140c469ed908bd1913fb9df05e20e52f3c7067e7c2680b43883796426"} Jul 18 02:41:53.762670 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:53.762228 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" event={"ID":"85840a40-e5f9-49a6-9356-55f239d63b4f","Type":"ContainerStarted","Data":"ed267982a5ceebf89205045f6e043a11d92fa9f95df7c4982b7db1643d587260"} Jul 18 02:41:53.779610 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:53.779563 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" podStartSLOduration=1.779547783 podStartE2EDuration="1.779547783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:53.776834513 +0000 UTC m=+2701.565171107" watchObservedRunningTime="2026-07-18 02:41:53.779547783 +0000 UTC m=+2701.567884364" Jul 18 02:41:56.102656 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:56.102617 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:41:56.202890 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:41:56.202859 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:42:02.945621 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:02.945577 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:42:02.945621 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:02.945628 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:42:02.958496 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:02.958472 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:42:03.826398 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:03.826369 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:42:13.117384 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:13.117345 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:42:26.114455 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:26.114419 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:42:56.970391 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:56.970350 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:42:57.926134 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:57.926100 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th"] Jul 18 02:42:57.946045 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:57.946016 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:42:58.849861 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:58.849827 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7689-140492067864112"] Jul 18 02:42:58.853738 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:58.853718 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492067864112" Jul 18 02:42:58.856478 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:58.856456 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tpsmz\"" Jul 18 02:42:58.858921 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:58.858900 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492067864112"] Jul 18 02:42:58.961553 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:58.961514 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9n6v\" (UniqueName: \"kubernetes.io/projected/54091071-139d-4b65-97d6-407cbb21611f-kube-api-access-x9n6v\") pod \"test-curl-7689-140492067864112\" (UID: \"54091071-139d-4b65-97d6-407cbb21611f\") " pod="opendatahub/test-curl-7689-140492067864112" Jul 18 02:42:59.062860 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:59.062823 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9n6v\" (UniqueName: \"kubernetes.io/projected/54091071-139d-4b65-97d6-407cbb21611f-kube-api-access-x9n6v\") pod \"test-curl-7689-140492067864112\" (UID: \"54091071-139d-4b65-97d6-407cbb21611f\") " pod="opendatahub/test-curl-7689-140492067864112" Jul 18 02:42:59.071351 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:59.071318 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9n6v\" (UniqueName: \"kubernetes.io/projected/54091071-139d-4b65-97d6-407cbb21611f-kube-api-access-x9n6v\") pod \"test-curl-7689-140492067864112\" (UID: \"54091071-139d-4b65-97d6-407cbb21611f\") " pod="opendatahub/test-curl-7689-140492067864112" Jul 18 02:42:59.164946 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:59.164905 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492067864112" Jul 18 02:42:59.296161 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:59.296129 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492067864112"] Jul 18 02:42:59.301621 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:42:59.301596 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54091071_139d_4b65_97d6_407cbb21611f.slice/crio-71ce0cc349463c0ac3ad55f201715bba7d5d16e176203467deaa1c486a4f081f WatchSource:0}: Error finding container 71ce0cc349463c0ac3ad55f201715bba7d5d16e176203467deaa1c486a4f081f: Status 404 returned error can't find the container with id 71ce0cc349463c0ac3ad55f201715bba7d5d16e176203467deaa1c486a4f081f Jul 18 02:42:59.302716 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:42:59.302696 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:43:00.089366 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:00.089339 2573 generic.go:356] "Generic (PLEG): container finished" podID="54091071-139d-4b65-97d6-407cbb21611f" containerID="50ecfbbeb3cd778dbcd8e31a73d1d201a50efe5e7e9608b285dda2b017db1e98" exitCode=0 Jul 18 02:43:00.089707 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:00.089400 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492067864112" event={"ID":"54091071-139d-4b65-97d6-407cbb21611f","Type":"ContainerDied","Data":"50ecfbbeb3cd778dbcd8e31a73d1d201a50efe5e7e9608b285dda2b017db1e98"} Jul 18 02:43:00.089707 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:00.089438 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492067864112" event={"ID":"54091071-139d-4b65-97d6-407cbb21611f","Type":"ContainerStarted","Data":"71ce0cc349463c0ac3ad55f201715bba7d5d16e176203467deaa1c486a4f081f"} Jul 18 02:43:01.225143 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:01.225118 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492067864112" Jul 18 02:43:01.272509 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:01.272480 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7689-140492067864112_54091071-139d-4b65-97d6-407cbb21611f/test-curl-7689-140492067864112/0.log" Jul 18 02:43:01.284774 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:01.284747 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9n6v\" (UniqueName: \"kubernetes.io/projected/54091071-139d-4b65-97d6-407cbb21611f-kube-api-access-x9n6v\") pod \"54091071-139d-4b65-97d6-407cbb21611f\" (UID: \"54091071-139d-4b65-97d6-407cbb21611f\") " Jul 18 02:43:01.286850 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:01.286826 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54091071-139d-4b65-97d6-407cbb21611f-kube-api-access-x9n6v" (OuterVolumeSpecName: "kube-api-access-x9n6v") pod "54091071-139d-4b65-97d6-407cbb21611f" (UID: "54091071-139d-4b65-97d6-407cbb21611f"). InnerVolumeSpecName "kube-api-access-x9n6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:01.315852 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:01.315818 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7689-140492067864112"] Jul 18 02:43:01.325784 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:01.325757 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7689-140492067864112"] Jul 18 02:43:01.385388 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:01.385351 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9n6v\" (UniqueName: \"kubernetes.io/projected/54091071-139d-4b65-97d6-407cbb21611f-kube-api-access-x9n6v\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:02.106191 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:02.106162 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492067864112" Jul 18 02:43:02.106381 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:02.106200 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71ce0cc349463c0ac3ad55f201715bba7d5d16e176203467deaa1c486a4f081f" Jul 18 02:43:02.116360 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:43:02.116332 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7689-140492067864112\" is forbidden: User \"system:node:ip-10-0-130-187.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-187.ec2.internal' and this object" podUID="54091071-139d-4b65-97d6-407cbb21611f" pod="opendatahub/test-curl-7689-140492067864112" Jul 18 02:43:02.721214 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:43:02.721174 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7689-140492067864112\" is forbidden: User \"system:node:ip-10-0-130-187.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-187.ec2.internal' and this object" podUID="54091071-139d-4b65-97d6-407cbb21611f" pod="opendatahub/test-curl-7689-140492067864112" Jul 18 02:43:02.722598 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:02.722576 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54091071-139d-4b65-97d6-407cbb21611f" path="/var/lib/kubelet/pods/54091071-139d-4b65-97d6-407cbb21611f/volumes" Jul 18 02:43:04.531949 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.531917 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7689-140492071449488"] Jul 18 02:43:04.532338 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.532244 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54091071-139d-4b65-97d6-407cbb21611f" containerName="test-curl-7689-140492067864112" Jul 18 02:43:04.532338 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.532256 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="54091071-139d-4b65-97d6-407cbb21611f" containerName="test-curl-7689-140492067864112" Jul 18 02:43:04.532338 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.532323 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="54091071-139d-4b65-97d6-407cbb21611f" containerName="test-curl-7689-140492067864112" Jul 18 02:43:04.535182 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.535166 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:04.537352 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.537332 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tpsmz\"" Jul 18 02:43:04.542558 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.542536 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492071449488"] Jul 18 02:43:04.613532 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.613494 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlpqj\" (UniqueName: \"kubernetes.io/projected/f22f12ca-c953-45fd-bad5-52c2e386d8bf-kube-api-access-mlpqj\") pod \"test-curl-7689-140492071449488\" (UID: \"f22f12ca-c953-45fd-bad5-52c2e386d8bf\") " pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:04.714607 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.714572 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlpqj\" (UniqueName: \"kubernetes.io/projected/f22f12ca-c953-45fd-bad5-52c2e386d8bf-kube-api-access-mlpqj\") pod \"test-curl-7689-140492071449488\" (UID: \"f22f12ca-c953-45fd-bad5-52c2e386d8bf\") " pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:04.722472 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.722448 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlpqj\" (UniqueName: \"kubernetes.io/projected/f22f12ca-c953-45fd-bad5-52c2e386d8bf-kube-api-access-mlpqj\") pod \"test-curl-7689-140492071449488\" (UID: \"f22f12ca-c953-45fd-bad5-52c2e386d8bf\") " pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:04.845031 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.844941 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:04.974840 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:04.974809 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492071449488"] Jul 18 02:43:04.975542 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:43:04.975517 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf22f12ca_c953_45fd_bad5_52c2e386d8bf.slice/crio-ac52adcb4c9d13ac3b47bd07cc94be11a190aebd0ca21a19c1c7993ddc356a45 WatchSource:0}: Error finding container ac52adcb4c9d13ac3b47bd07cc94be11a190aebd0ca21a19c1c7993ddc356a45: Status 404 returned error can't find the container with id ac52adcb4c9d13ac3b47bd07cc94be11a190aebd0ca21a19c1c7993ddc356a45 Jul 18 02:43:05.124545 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:05.124514 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492071449488" event={"ID":"f22f12ca-c953-45fd-bad5-52c2e386d8bf","Type":"ContainerStarted","Data":"ac52adcb4c9d13ac3b47bd07cc94be11a190aebd0ca21a19c1c7993ddc356a45"} Jul 18 02:43:06.130808 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:06.130768 2573 generic.go:356] "Generic (PLEG): container finished" podID="f22f12ca-c953-45fd-bad5-52c2e386d8bf" containerID="3a67054c4dac4ae17f01a24e3f2366d3bc96690b5704a3abfb739ec1e3098669" exitCode=0 Jul 18 02:43:06.131235 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:06.130847 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492071449488" event={"ID":"f22f12ca-c953-45fd-bad5-52c2e386d8bf","Type":"ContainerDied","Data":"3a67054c4dac4ae17f01a24e3f2366d3bc96690b5704a3abfb739ec1e3098669"} Jul 18 02:43:07.265734 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.265708 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:07.284063 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.284036 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7689-140492071449488_f22f12ca-c953-45fd-bad5-52c2e386d8bf/test-curl-7689-140492071449488/0.log" Jul 18 02:43:07.306737 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.306705 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7689-140492071449488"] Jul 18 02:43:07.310269 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.310248 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7689-140492071449488"] Jul 18 02:43:07.336477 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.336444 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlpqj\" (UniqueName: \"kubernetes.io/projected/f22f12ca-c953-45fd-bad5-52c2e386d8bf-kube-api-access-mlpqj\") pod \"f22f12ca-c953-45fd-bad5-52c2e386d8bf\" (UID: \"f22f12ca-c953-45fd-bad5-52c2e386d8bf\") " Jul 18 02:43:07.338466 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.338438 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f22f12ca-c953-45fd-bad5-52c2e386d8bf-kube-api-access-mlpqj" (OuterVolumeSpecName: "kube-api-access-mlpqj") pod "f22f12ca-c953-45fd-bad5-52c2e386d8bf" (UID: "f22f12ca-c953-45fd-bad5-52c2e386d8bf"). InnerVolumeSpecName "kube-api-access-mlpqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:07.418093 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.418019 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7689-140492071449488"] Jul 18 02:43:07.418374 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.418362 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f22f12ca-c953-45fd-bad5-52c2e386d8bf" containerName="test-curl-7689-140492071449488" Jul 18 02:43:07.418427 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.418375 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f22f12ca-c953-45fd-bad5-52c2e386d8bf" containerName="test-curl-7689-140492071449488" Jul 18 02:43:07.418464 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.418429 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f22f12ca-c953-45fd-bad5-52c2e386d8bf" containerName="test-curl-7689-140492071449488" Jul 18 02:43:07.421187 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.421172 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:07.428118 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.428071 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492071449488"] Jul 18 02:43:07.437205 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.437176 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlpqj\" (UniqueName: \"kubernetes.io/projected/f22f12ca-c953-45fd-bad5-52c2e386d8bf-kube-api-access-mlpqj\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:07.538360 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.538331 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mlmh\" (UniqueName: \"kubernetes.io/projected/f610947b-ae25-43b3-bced-7098f24c8aa9-kube-api-access-2mlmh\") pod \"test-curl-7689-140492071449488\" (UID: \"f610947b-ae25-43b3-bced-7098f24c8aa9\") " pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:07.638936 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.638904 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mlmh\" (UniqueName: \"kubernetes.io/projected/f610947b-ae25-43b3-bced-7098f24c8aa9-kube-api-access-2mlmh\") pod \"test-curl-7689-140492071449488\" (UID: \"f610947b-ae25-43b3-bced-7098f24c8aa9\") " pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:07.646671 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.646647 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mlmh\" (UniqueName: \"kubernetes.io/projected/f610947b-ae25-43b3-bced-7098f24c8aa9-kube-api-access-2mlmh\") pod \"test-curl-7689-140492071449488\" (UID: \"f610947b-ae25-43b3-bced-7098f24c8aa9\") " pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:07.730804 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.730731 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:07.854447 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:07.854421 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492071449488"] Jul 18 02:43:07.856835 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:43:07.856812 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf610947b_ae25_43b3_bced_7098f24c8aa9.slice/crio-d7886892f538b8106befce5e77ced4be7c8a106be0fe7241d5e33ac63200e4f5 WatchSource:0}: Error finding container d7886892f538b8106befce5e77ced4be7c8a106be0fe7241d5e33ac63200e4f5: Status 404 returned error can't find the container with id d7886892f538b8106befce5e77ced4be7c8a106be0fe7241d5e33ac63200e4f5 Jul 18 02:43:08.147252 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:08.147226 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492071449488" event={"ID":"f610947b-ae25-43b3-bced-7098f24c8aa9","Type":"ContainerStarted","Data":"d7886892f538b8106befce5e77ced4be7c8a106be0fe7241d5e33ac63200e4f5"} Jul 18 02:43:08.148506 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:08.148487 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac52adcb4c9d13ac3b47bd07cc94be11a190aebd0ca21a19c1c7993ddc356a45" Jul 18 02:43:08.148589 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:08.148495 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:08.159462 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:08.159433 2573 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7689-140492071449488" oldPodUID="f22f12ca-c953-45fd-bad5-52c2e386d8bf" podUID="f610947b-ae25-43b3-bced-7098f24c8aa9" Jul 18 02:43:08.723653 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:08.723611 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f22f12ca-c953-45fd-bad5-52c2e386d8bf" path="/var/lib/kubelet/pods/f22f12ca-c953-45fd-bad5-52c2e386d8bf/volumes" Jul 18 02:43:09.154762 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:09.154726 2573 generic.go:356] "Generic (PLEG): container finished" podID="f610947b-ae25-43b3-bced-7098f24c8aa9" containerID="d1118a4dc41e58adde33aa8def58a38fe2d74ec0d9010bc288f0810dbb2441e4" exitCode=0 Jul 18 02:43:09.154943 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:09.154804 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492071449488" event={"ID":"f610947b-ae25-43b3-bced-7098f24c8aa9","Type":"ContainerDied","Data":"d1118a4dc41e58adde33aa8def58a38fe2d74ec0d9010bc288f0810dbb2441e4"} Jul 18 02:43:10.285282 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.285253 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:10.306096 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.306064 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7689-140492071449488_f610947b-ae25-43b3-bced-7098f24c8aa9/test-curl-7689-140492071449488/0.log" Jul 18 02:43:10.327948 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.327924 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7689-140492071449488"] Jul 18 02:43:10.333904 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.333880 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7689-140492071449488"] Jul 18 02:43:10.362388 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.362363 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mlmh\" (UniqueName: \"kubernetes.io/projected/f610947b-ae25-43b3-bced-7098f24c8aa9-kube-api-access-2mlmh\") pod \"f610947b-ae25-43b3-bced-7098f24c8aa9\" (UID: \"f610947b-ae25-43b3-bced-7098f24c8aa9\") " Jul 18 02:43:10.364305 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.364278 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f610947b-ae25-43b3-bced-7098f24c8aa9-kube-api-access-2mlmh" (OuterVolumeSpecName: "kube-api-access-2mlmh") pod "f610947b-ae25-43b3-bced-7098f24c8aa9" (UID: "f610947b-ae25-43b3-bced-7098f24c8aa9"). InnerVolumeSpecName "kube-api-access-2mlmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:10.449426 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.449350 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7689-140492071450496"] Jul 18 02:43:10.449678 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.449666 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f610947b-ae25-43b3-bced-7098f24c8aa9" containerName="test-curl-7689-140492071449488" Jul 18 02:43:10.449723 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.449679 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f610947b-ae25-43b3-bced-7098f24c8aa9" containerName="test-curl-7689-140492071449488" Jul 18 02:43:10.449763 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.449734 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f610947b-ae25-43b3-bced-7098f24c8aa9" containerName="test-curl-7689-140492071449488" Jul 18 02:43:10.452446 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.452429 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:10.458588 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.458564 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492071450496"] Jul 18 02:43:10.463692 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.463672 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mlmh\" (UniqueName: \"kubernetes.io/projected/f610947b-ae25-43b3-bced-7098f24c8aa9-kube-api-access-2mlmh\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:10.565046 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.565015 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kqgd\" (UniqueName: \"kubernetes.io/projected/a157d1bb-eb2e-4df5-99d5-310453ce018c-kube-api-access-6kqgd\") pod \"test-curl-7689-140492071450496\" (UID: \"a157d1bb-eb2e-4df5-99d5-310453ce018c\") " pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:10.666340 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.666305 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kqgd\" (UniqueName: \"kubernetes.io/projected/a157d1bb-eb2e-4df5-99d5-310453ce018c-kube-api-access-6kqgd\") pod \"test-curl-7689-140492071450496\" (UID: \"a157d1bb-eb2e-4df5-99d5-310453ce018c\") " pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:10.676148 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.676123 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kqgd\" (UniqueName: \"kubernetes.io/projected/a157d1bb-eb2e-4df5-99d5-310453ce018c-kube-api-access-6kqgd\") pod \"test-curl-7689-140492071450496\" (UID: \"a157d1bb-eb2e-4df5-99d5-310453ce018c\") " pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:10.729217 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.729129 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f610947b-ae25-43b3-bced-7098f24c8aa9" path="/var/lib/kubelet/pods/f610947b-ae25-43b3-bced-7098f24c8aa9/volumes" Jul 18 02:43:10.762203 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.762181 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:10.888758 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:10.888725 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492071450496"] Jul 18 02:43:10.890276 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:43:10.890243 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda157d1bb_eb2e_4df5_99d5_310453ce018c.slice/crio-033e8bebaf88ec7cd8120c5dde69d35709d863bfc2f7ce72bb3b9a4e40880a94 WatchSource:0}: Error finding container 033e8bebaf88ec7cd8120c5dde69d35709d863bfc2f7ce72bb3b9a4e40880a94: Status 404 returned error can't find the container with id 033e8bebaf88ec7cd8120c5dde69d35709d863bfc2f7ce72bb3b9a4e40880a94 Jul 18 02:43:11.165119 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:11.165073 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492071450496" event={"ID":"a157d1bb-eb2e-4df5-99d5-310453ce018c","Type":"ContainerStarted","Data":"033e8bebaf88ec7cd8120c5dde69d35709d863bfc2f7ce72bb3b9a4e40880a94"} Jul 18 02:43:11.166258 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:11.166233 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071449488" Jul 18 02:43:11.166395 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:11.166257 2573 scope.go:117] "RemoveContainer" containerID="d1118a4dc41e58adde33aa8def58a38fe2d74ec0d9010bc288f0810dbb2441e4" Jul 18 02:43:12.172807 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:12.172776 2573 generic.go:356] "Generic (PLEG): container finished" podID="a157d1bb-eb2e-4df5-99d5-310453ce018c" containerID="2db8d78336a10ccc5418ad5d22259b9195446577f095161df82a568c94c07594" exitCode=0 Jul 18 02:43:12.173258 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:12.172864 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492071450496" event={"ID":"a157d1bb-eb2e-4df5-99d5-310453ce018c","Type":"ContainerDied","Data":"2db8d78336a10ccc5418ad5d22259b9195446577f095161df82a568c94c07594"} Jul 18 02:43:12.945589 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:12.945550 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" podUID="85840a40-e5f9-49a6-9356-55f239d63b4f" containerName="main" containerID="cri-o://7782d4e140c469ed908bd1913fb9df05e20e52f3c7067e7c2680b43883796426" gracePeriod=45 Jul 18 02:43:13.180402 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.180375 2573 generic.go:356] "Generic (PLEG): container finished" podID="85840a40-e5f9-49a6-9356-55f239d63b4f" containerID="7782d4e140c469ed908bd1913fb9df05e20e52f3c7067e7c2680b43883796426" exitCode=0 Jul 18 02:43:13.180750 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.180451 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" event={"ID":"85840a40-e5f9-49a6-9356-55f239d63b4f","Type":"ContainerDied","Data":"7782d4e140c469ed908bd1913fb9df05e20e52f3c7067e7c2680b43883796426"} Jul 18 02:43:13.197688 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.197626 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:43:13.291434 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.291402 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-dshm\") pod \"85840a40-e5f9-49a6-9356-55f239d63b4f\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " Jul 18 02:43:13.291611 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.291492 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27gm5\" (UniqueName: \"kubernetes.io/projected/85840a40-e5f9-49a6-9356-55f239d63b4f-kube-api-access-27gm5\") pod \"85840a40-e5f9-49a6-9356-55f239d63b4f\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " Jul 18 02:43:13.291611 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.291521 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-model-cache\") pod \"85840a40-e5f9-49a6-9356-55f239d63b4f\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " Jul 18 02:43:13.291611 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.291581 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85840a40-e5f9-49a6-9356-55f239d63b4f-tls-certs\") pod \"85840a40-e5f9-49a6-9356-55f239d63b4f\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " Jul 18 02:43:13.291611 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.291597 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-tmp-dir\") pod \"85840a40-e5f9-49a6-9356-55f239d63b4f\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " Jul 18 02:43:13.291611 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.291611 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-home\") pod \"85840a40-e5f9-49a6-9356-55f239d63b4f\" (UID: \"85840a40-e5f9-49a6-9356-55f239d63b4f\") " Jul 18 02:43:13.291897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.291872 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "85840a40-e5f9-49a6-9356-55f239d63b4f" (UID: "85840a40-e5f9-49a6-9356-55f239d63b4f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:13.292014 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.291953 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-model-cache" (OuterVolumeSpecName: "model-cache") pod "85840a40-e5f9-49a6-9356-55f239d63b4f" (UID: "85840a40-e5f9-49a6-9356-55f239d63b4f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:13.292094 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.292044 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-home" (OuterVolumeSpecName: "home") pod "85840a40-e5f9-49a6-9356-55f239d63b4f" (UID: "85840a40-e5f9-49a6-9356-55f239d63b4f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:13.293645 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.293624 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85840a40-e5f9-49a6-9356-55f239d63b4f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "85840a40-e5f9-49a6-9356-55f239d63b4f" (UID: "85840a40-e5f9-49a6-9356-55f239d63b4f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:43:13.293867 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.293853 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-dshm" (OuterVolumeSpecName: "dshm") pod "85840a40-e5f9-49a6-9356-55f239d63b4f" (UID: "85840a40-e5f9-49a6-9356-55f239d63b4f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:13.293954 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.293869 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85840a40-e5f9-49a6-9356-55f239d63b4f-kube-api-access-27gm5" (OuterVolumeSpecName: "kube-api-access-27gm5") pod "85840a40-e5f9-49a6-9356-55f239d63b4f" (UID: "85840a40-e5f9-49a6-9356-55f239d63b4f"). InnerVolumeSpecName "kube-api-access-27gm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:13.315495 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.315475 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:13.335322 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.335294 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7689-140492071450496_a157d1bb-eb2e-4df5-99d5-310453ce018c/test-curl-7689-140492071450496/0.log" Jul 18 02:43:13.357883 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.357851 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7689-140492071450496"] Jul 18 02:43:13.360156 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.360131 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7689-140492071450496"] Jul 18 02:43:13.392597 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.392571 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kqgd\" (UniqueName: \"kubernetes.io/projected/a157d1bb-eb2e-4df5-99d5-310453ce018c-kube-api-access-6kqgd\") pod \"a157d1bb-eb2e-4df5-99d5-310453ce018c\" (UID: \"a157d1bb-eb2e-4df5-99d5-310453ce018c\") " Jul 18 02:43:13.392828 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.392810 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27gm5\" (UniqueName: \"kubernetes.io/projected/85840a40-e5f9-49a6-9356-55f239d63b4f-kube-api-access-27gm5\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:13.392884 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.392831 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-model-cache\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:13.392884 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.392841 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85840a40-e5f9-49a6-9356-55f239d63b4f-tls-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:13.392884 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.392871 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-tmp-dir\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:13.392884 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.392879 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-home\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:13.393050 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.392886 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85840a40-e5f9-49a6-9356-55f239d63b4f-dshm\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:13.394371 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.394351 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a157d1bb-eb2e-4df5-99d5-310453ce018c-kube-api-access-6kqgd" (OuterVolumeSpecName: "kube-api-access-6kqgd") pod "a157d1bb-eb2e-4df5-99d5-310453ce018c" (UID: "a157d1bb-eb2e-4df5-99d5-310453ce018c"). InnerVolumeSpecName "kube-api-access-6kqgd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:13.477276 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.477220 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7689-140492071450496"] Jul 18 02:43:13.477535 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.477523 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85840a40-e5f9-49a6-9356-55f239d63b4f" containerName="main" Jul 18 02:43:13.477535 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.477536 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="85840a40-e5f9-49a6-9356-55f239d63b4f" containerName="main" Jul 18 02:43:13.477604 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.477545 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a157d1bb-eb2e-4df5-99d5-310453ce018c" containerName="test-curl-7689-140492071450496" Jul 18 02:43:13.477604 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.477553 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a157d1bb-eb2e-4df5-99d5-310453ce018c" containerName="test-curl-7689-140492071450496" Jul 18 02:43:13.477669 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.477604 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="85840a40-e5f9-49a6-9356-55f239d63b4f" containerName="main" Jul 18 02:43:13.477669 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.477613 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a157d1bb-eb2e-4df5-99d5-310453ce018c" containerName="test-curl-7689-140492071450496" Jul 18 02:43:13.480256 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.480242 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:13.484172 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.484149 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492071450496"] Jul 18 02:43:13.494061 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.494037 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kqgd\" (UniqueName: \"kubernetes.io/projected/a157d1bb-eb2e-4df5-99d5-310453ce018c-kube-api-access-6kqgd\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:13.594440 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.594409 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkw5r\" (UniqueName: \"kubernetes.io/projected/fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb-kube-api-access-zkw5r\") pod \"test-curl-7689-140492071450496\" (UID: \"fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb\") " pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:13.695776 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.695744 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkw5r\" (UniqueName: \"kubernetes.io/projected/fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb-kube-api-access-zkw5r\") pod \"test-curl-7689-140492071450496\" (UID: \"fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb\") " pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:13.704407 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.704379 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkw5r\" (UniqueName: \"kubernetes.io/projected/fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb-kube-api-access-zkw5r\") pod \"test-curl-7689-140492071450496\" (UID: \"fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb\") " pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:13.789912 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.789845 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:13.916288 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:13.916151 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492071450496"] Jul 18 02:43:13.918423 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:43:13.918390 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbf60d7d_a727_4132_ab1d_d2bdbe7a68bb.slice/crio-ea28d253f24fb1c39f55bcc9c250fbc3f5dbff5e3fd79d9e756bfcb1dc7654ae WatchSource:0}: Error finding container ea28d253f24fb1c39f55bcc9c250fbc3f5dbff5e3fd79d9e756bfcb1dc7654ae: Status 404 returned error can't find the container with id ea28d253f24fb1c39f55bcc9c250fbc3f5dbff5e3fd79d9e756bfcb1dc7654ae Jul 18 02:43:14.190022 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.189996 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" Jul 18 02:43:14.190022 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.190009 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th" event={"ID":"85840a40-e5f9-49a6-9356-55f239d63b4f","Type":"ContainerDied","Data":"ed267982a5ceebf89205045f6e043a11d92fa9f95df7c4982b7db1643d587260"} Jul 18 02:43:14.190537 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.190066 2573 scope.go:117] "RemoveContainer" containerID="7782d4e140c469ed908bd1913fb9df05e20e52f3c7067e7c2680b43883796426" Jul 18 02:43:14.191486 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.191466 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:14.191486 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.191481 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="033e8bebaf88ec7cd8120c5dde69d35709d863bfc2f7ce72bb3b9a4e40880a94" Jul 18 02:43:14.194892 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.194846 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492071450496" event={"ID":"fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb","Type":"ContainerStarted","Data":"ea28d253f24fb1c39f55bcc9c250fbc3f5dbff5e3fd79d9e756bfcb1dc7654ae"} Jul 18 02:43:14.216581 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.216555 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th"] Jul 18 02:43:14.219784 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.219758 2573 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7689-140492071450496" oldPodUID="a157d1bb-eb2e-4df5-99d5-310453ce018c" podUID="fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb" Jul 18 02:43:14.220134 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.220114 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-922cd9/rate-test-model-922cd9-kserve-6875699f-lx8th"] Jul 18 02:43:14.723256 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.723213 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85840a40-e5f9-49a6-9356-55f239d63b4f" path="/var/lib/kubelet/pods/85840a40-e5f9-49a6-9356-55f239d63b4f/volumes" Jul 18 02:43:14.723614 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:14.723596 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a157d1bb-eb2e-4df5-99d5-310453ce018c" path="/var/lib/kubelet/pods/a157d1bb-eb2e-4df5-99d5-310453ce018c/volumes" Jul 18 02:43:15.200739 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:15.200700 2573 generic.go:356] "Generic (PLEG): container finished" podID="fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb" containerID="068dafdc2ee032b2580c60bd90c07c464915178a19c59c6fe872fddf4eb46cdd" exitCode=0 Jul 18 02:43:15.201219 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:15.200792 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492071450496" event={"ID":"fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb","Type":"ContainerDied","Data":"068dafdc2ee032b2580c60bd90c07c464915178a19c59c6fe872fddf4eb46cdd"} Jul 18 02:43:16.340563 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:16.340542 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:16.361572 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:16.361546 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7689-140492071450496_fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb/test-curl-7689-140492071450496/0.log" Jul 18 02:43:16.383649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:16.383623 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7689-140492071450496"] Jul 18 02:43:16.387305 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:16.387282 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7689-140492071450496"] Jul 18 02:43:16.419225 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:16.419201 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkw5r\" (UniqueName: \"kubernetes.io/projected/fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb-kube-api-access-zkw5r\") pod \"fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb\" (UID: \"fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb\") " Jul 18 02:43:16.421193 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:16.421167 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb-kube-api-access-zkw5r" (OuterVolumeSpecName: "kube-api-access-zkw5r") pod "fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb" (UID: "fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb"). InnerVolumeSpecName "kube-api-access-zkw5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:16.519924 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:16.519856 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkw5r\" (UniqueName: \"kubernetes.io/projected/fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb-kube-api-access-zkw5r\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:16.722145 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:16.722114 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb" path="/var/lib/kubelet/pods/fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb/volumes" Jul 18 02:43:17.081278 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:17.081248 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:17.180986 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:17.180941 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:17.211881 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:17.211859 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492071450496" Jul 18 02:43:17.212041 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:17.211920 2573 scope.go:117] "RemoveContainer" containerID="068dafdc2ee032b2580c60bd90c07c464915178a19c59c6fe872fddf4eb46cdd" Jul 18 02:43:26.990526 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:26.990488 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:31.637174 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:31.637138 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:32.027117 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:32.027038 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:32.086337 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:32.086284 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:37.743983 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:37.743928 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:37.841354 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:37.841306 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:37.971398 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:37.971365 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs"] Jul 18 02:43:37.971682 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:37.971658 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" containerID="cri-o://b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63" gracePeriod=30 Jul 18 02:43:37.998665 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:37.998595 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:38.212915 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:38.212881 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:38.855365 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:38.855335 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:38.951734 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:38.951704 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:43.216451 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.216416 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:43:43.228134 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228110 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-socket\") pod \"621de3d6-205d-44cb-a382-bd2a61c82d50\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " Jul 18 02:43:43.228228 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228167 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-credential-socket\") pod \"621de3d6-205d-44cb-a382-bd2a61c82d50\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " Jul 18 02:43:43.228228 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228190 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-podinfo\") pod \"621de3d6-205d-44cb-a382-bd2a61c82d50\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " Jul 18 02:43:43.228228 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228220 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-envoy\") pod \"621de3d6-205d-44cb-a382-bd2a61c82d50\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " Jul 18 02:43:43.228376 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228234 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-token\") pod \"621de3d6-205d-44cb-a382-bd2a61c82d50\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " Jul 18 02:43:43.228376 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228266 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621de3d6-205d-44cb-a382-bd2a61c82d50-istiod-ca-cert\") pod \"621de3d6-205d-44cb-a382-bd2a61c82d50\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " Jul 18 02:43:43.228376 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228307 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-data\") pod \"621de3d6-205d-44cb-a382-bd2a61c82d50\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " Jul 18 02:43:43.228376 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228312 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "621de3d6-205d-44cb-a382-bd2a61c82d50" (UID: "621de3d6-205d-44cb-a382-bd2a61c82d50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:43.228376 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228330 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt9g5\" (UniqueName: \"kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-kube-api-access-rt9g5\") pod \"621de3d6-205d-44cb-a382-bd2a61c82d50\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " Jul 18 02:43:43.228376 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228358 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-certs\") pod \"621de3d6-205d-44cb-a382-bd2a61c82d50\" (UID: \"621de3d6-205d-44cb-a382-bd2a61c82d50\") " Jul 18 02:43:43.228687 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228434 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "621de3d6-205d-44cb-a382-bd2a61c82d50" (UID: "621de3d6-205d-44cb-a382-bd2a61c82d50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:43.228687 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228677 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-credential-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:43.228789 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228696 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-socket\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:43.228789 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228717 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "621de3d6-205d-44cb-a382-bd2a61c82d50" (UID: "621de3d6-205d-44cb-a382-bd2a61c82d50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:43.228789 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.228734 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/621de3d6-205d-44cb-a382-bd2a61c82d50-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "621de3d6-205d-44cb-a382-bd2a61c82d50" (UID: "621de3d6-205d-44cb-a382-bd2a61c82d50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:43:43.229567 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.229539 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-data" (OuterVolumeSpecName: "istio-data") pod "621de3d6-205d-44cb-a382-bd2a61c82d50" (UID: "621de3d6-205d-44cb-a382-bd2a61c82d50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:43.230564 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.230530 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-token" (OuterVolumeSpecName: "istio-token") pod "621de3d6-205d-44cb-a382-bd2a61c82d50" (UID: "621de3d6-205d-44cb-a382-bd2a61c82d50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:43.230674 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.230586 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "621de3d6-205d-44cb-a382-bd2a61c82d50" (UID: "621de3d6-205d-44cb-a382-bd2a61c82d50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:43.230674 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.230597 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "621de3d6-205d-44cb-a382-bd2a61c82d50" (UID: "621de3d6-205d-44cb-a382-bd2a61c82d50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:43:43.230674 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.230596 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-kube-api-access-rt9g5" (OuterVolumeSpecName: "kube-api-access-rt9g5") pod "621de3d6-205d-44cb-a382-bd2a61c82d50" (UID: "621de3d6-205d-44cb-a382-bd2a61c82d50"). InnerVolumeSpecName "kube-api-access-rt9g5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:43.329716 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.329632 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-podinfo\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:43.329716 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.329661 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-envoy\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:43.329716 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.329670 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-token\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:43.329716 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.329678 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621de3d6-205d-44cb-a382-bd2a61c82d50-istiod-ca-cert\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:43.329716 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.329685 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-istio-data\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:43.329716 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.329694 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt9g5\" (UniqueName: \"kubernetes.io/projected/621de3d6-205d-44cb-a382-bd2a61c82d50-kube-api-access-rt9g5\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:43.329716 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.329703 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621de3d6-205d-44cb-a382-bd2a61c82d50-workload-certs\") on node \"ip-10-0-130-187.ec2.internal\" DevicePath \"\"" Jul 18 02:43:43.339685 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.339657 2573 generic.go:356] "Generic (PLEG): container finished" podID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerID="b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63" exitCode=0 Jul 18 02:43:43.339819 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.339710 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" Jul 18 02:43:43.339819 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.339711 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" event={"ID":"621de3d6-205d-44cb-a382-bd2a61c82d50","Type":"ContainerDied","Data":"b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63"} Jul 18 02:43:43.339893 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.339833 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs" event={"ID":"621de3d6-205d-44cb-a382-bd2a61c82d50","Type":"ContainerDied","Data":"b9eeb666c147dbed7048de71f495c63846e636f17ebfb1dd0485dbce520d94ec"} Jul 18 02:43:43.339893 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.339864 2573 scope.go:117] "RemoveContainer" containerID="b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63" Jul 18 02:43:43.349257 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.349235 2573 scope.go:117] "RemoveContainer" containerID="0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23" Jul 18 02:43:43.357128 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.357111 2573 scope.go:117] "RemoveContainer" containerID="b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63" Jul 18 02:43:43.357360 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:43:43.357339 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63\": container with ID starting with b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63 not found: ID does not exist" containerID="b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63" Jul 18 02:43:43.357416 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.357367 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63"} err="failed to get container status \"b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63\": rpc error: code = NotFound desc = could not find container \"b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63\": container with ID starting with b83171113a335ce863860aaaeb978115283d560a0150dc3d1386ce7ffd161a63 not found: ID does not exist" Jul 18 02:43:43.357416 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.357383 2573 scope.go:117] "RemoveContainer" containerID="0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23" Jul 18 02:43:43.357598 ip-10-0-130-187 kubenswrapper[2573]: E0718 02:43:43.357578 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23\": container with ID starting with 0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23 not found: ID does not exist" containerID="0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23" Jul 18 02:43:43.357639 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.357596 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23"} err="failed to get container status \"0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23\": rpc error: code = NotFound desc = could not find container \"0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23\": container with ID starting with 0acfd373be665b4418a3b474fd66292186f9f57f364317a11173e3edfd127f23 not found: ID does not exist" Jul 18 02:43:43.361564 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.361543 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs"] Jul 18 02:43:43.367361 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:43.367341 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-0d6b43-openshift-default-b4ff65755-hdtjs"] Jul 18 02:43:44.722500 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:44.722469 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" path="/var/lib/kubelet/pods/621de3d6-205d-44cb-a382-bd2a61c82d50/volumes" Jul 18 02:43:48.851101 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:48.851065 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:53.441981 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:53.441930 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:43:53.819024 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:43:53.818920 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:44:00.105168 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:00.105129 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:44:00.207928 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:00.207893 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:44:00.373935 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:00.373906 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:44:00.641800 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:00.641707 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:44:01.073119 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:01.073030 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-568fcc8d45-5r86j_b0269a7b-591d-4c64-889c-1dcb1660eadf/authorino/0.log" Jul 18 02:44:07.005767 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:07.005737 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-hlgh6_081cb735-ba0b-4e64-910a-bb7e7ad6d55a/copy-manifests/0.log" Jul 18 02:44:07.012342 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:07.012321 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-hlgh6_081cb735-ba0b-4e64-910a-bb7e7ad6d55a/manager/0.log" Jul 18 02:44:08.566672 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.566645 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h_de6109e1-4420-4de5-bff3-d3f957245170/util/0.log" Jul 18 02:44:08.572847 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.572823 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h_de6109e1-4420-4de5-bff3-d3f957245170/pull/0.log" Jul 18 02:44:08.579194 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.579176 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h_de6109e1-4420-4de5-bff3-d3f957245170/extract/0.log" Jul 18 02:44:08.688947 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.688923 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf_18429789-5590-4420-99cb-ada89afc4685/extract/0.log" Jul 18 02:44:08.694150 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.694132 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf_18429789-5590-4420-99cb-ada89afc4685/util/0.log" Jul 18 02:44:08.699823 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.699809 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf_18429789-5590-4420-99cb-ada89afc4685/pull/0.log" Jul 18 02:44:08.808793 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.808769 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm_b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22/util/0.log" Jul 18 02:44:08.814424 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.814401 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm_b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22/pull/0.log" Jul 18 02:44:08.819880 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.819830 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm_b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22/extract/0.log" Jul 18 02:44:08.934137 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.934116 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc_ccbd3263-6251-4e1c-8fe7-713e2fc39d53/util/0.log" Jul 18 02:44:08.939960 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.939942 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc_ccbd3263-6251-4e1c-8fe7-713e2fc39d53/pull/0.log" Jul 18 02:44:08.945787 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:08.945761 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc_ccbd3263-6251-4e1c-8fe7-713e2fc39d53/extract/0.log" Jul 18 02:44:09.069035 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:09.069010 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-568fcc8d45-5r86j_b0269a7b-591d-4c64-889c-1dcb1660eadf/authorino/0.log" Jul 18 02:44:09.186526 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:09.186501 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zkb2x_04dd269b-879b-4b6a-9ed8-6959092e82ee/manager/0.log" Jul 18 02:44:09.512918 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:09.512838 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-26s7w_2b249822-89e4-46af-93d4-370a6935bc53/registry-server/0.log" Jul 18 02:44:09.874319 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:09.874290 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9sntf_394487e0-7e76-4cd7-837b-c31265df51c4/manager/0.log" Jul 18 02:44:10.223517 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:10.223429 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4p724d_9e70ad11-ef7b-49b0-aafd-224381739a45/istio-proxy/0.log" Jul 18 02:44:10.688447 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:10.688416 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zt886_33381711-7e71-4b61-bf6a-e101f47a717b/istio-proxy/1.log" Jul 18 02:44:10.924660 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:10.924627 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-986hh_be269f87-e314-4ca5-bcf4-24658f1a4dee/payload-processing/0.log" Jul 18 02:44:11.484042 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:11.483999 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wft9l_b847ea02-1f8c-4b8e-b60d-6ca14611327d/main/0.log" Jul 18 02:44:11.820083 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:11.819990 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-4hzzk_2bc2d127-c82e-41e8-aff0-9d5a5fd3b3e8/main/0.log" Jul 18 02:44:17.697485 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.697449 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6vl8f/must-gather-gnbkn"] Jul 18 02:44:17.697866 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.697775 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" Jul 18 02:44:17.697866 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.697788 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" Jul 18 02:44:17.697866 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.697796 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb" containerName="test-curl-7689-140492071450496" Jul 18 02:44:17.697866 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.697802 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb" containerName="test-curl-7689-140492071450496" Jul 18 02:44:17.698029 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.697882 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" Jul 18 02:44:17.698029 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.697890 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf60d7d-a727-4132-ab1d-d2bdbe7a68bb" containerName="test-curl-7689-140492071450496" Jul 18 02:44:17.698029 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.697897 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" Jul 18 02:44:17.698029 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.697944 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" Jul 18 02:44:17.698029 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.697950 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="621de3d6-205d-44cb-a382-bd2a61c82d50" containerName="istio-proxy" Jul 18 02:44:17.701149 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.701128 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vl8f/must-gather-gnbkn" Jul 18 02:44:17.703445 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.703420 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6vl8f\"/\"openshift-service-ca.crt\"" Jul 18 02:44:17.703543 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.703484 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6vl8f\"/\"kube-root-ca.crt\"" Jul 18 02:44:17.715171 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.715148 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vl8f/must-gather-gnbkn"] Jul 18 02:44:17.722775 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.722748 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:44:17.799020 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.798959 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/162b89fb-f83e-4cb1-945e-41f67b39d01b-must-gather-output\") pod \"must-gather-gnbkn\" (UID: \"162b89fb-f83e-4cb1-945e-41f67b39d01b\") " pod="openshift-must-gather-6vl8f/must-gather-gnbkn" Jul 18 02:44:17.799020 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.799024 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbt29\" (UniqueName: \"kubernetes.io/projected/162b89fb-f83e-4cb1-945e-41f67b39d01b-kube-api-access-fbt29\") pod \"must-gather-gnbkn\" (UID: \"162b89fb-f83e-4cb1-945e-41f67b39d01b\") " pod="openshift-must-gather-6vl8f/must-gather-gnbkn" Jul 18 02:44:17.822499 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.822469 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" limit=1024 Jul 18 02:44:17.900079 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.900047 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/162b89fb-f83e-4cb1-945e-41f67b39d01b-must-gather-output\") pod \"must-gather-gnbkn\" (UID: \"162b89fb-f83e-4cb1-945e-41f67b39d01b\") " pod="openshift-must-gather-6vl8f/must-gather-gnbkn" Jul 18 02:44:17.900207 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.900081 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbt29\" (UniqueName: \"kubernetes.io/projected/162b89fb-f83e-4cb1-945e-41f67b39d01b-kube-api-access-fbt29\") pod \"must-gather-gnbkn\" (UID: \"162b89fb-f83e-4cb1-945e-41f67b39d01b\") " pod="openshift-must-gather-6vl8f/must-gather-gnbkn" Jul 18 02:44:17.900392 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.900371 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/162b89fb-f83e-4cb1-945e-41f67b39d01b-must-gather-output\") pod \"must-gather-gnbkn\" (UID: \"162b89fb-f83e-4cb1-945e-41f67b39d01b\") " pod="openshift-must-gather-6vl8f/must-gather-gnbkn" Jul 18 02:44:17.910578 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:17.910547 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbt29\" (UniqueName: \"kubernetes.io/projected/162b89fb-f83e-4cb1-945e-41f67b39d01b-kube-api-access-fbt29\") pod \"must-gather-gnbkn\" (UID: \"162b89fb-f83e-4cb1-945e-41f67b39d01b\") " pod="openshift-must-gather-6vl8f/must-gather-gnbkn" Jul 18 02:44:18.009857 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:18.009763 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vl8f/must-gather-gnbkn" Jul 18 02:44:18.135666 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:18.135641 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vl8f/must-gather-gnbkn"] Jul 18 02:44:18.137490 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:44:18.137465 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod162b89fb_f83e_4cb1_945e_41f67b39d01b.slice/crio-f924d2912ee8a5e57c1e2bbc84abfb2f6177dd6a256f75c52c2dd7994e18d1b2 WatchSource:0}: Error finding container f924d2912ee8a5e57c1e2bbc84abfb2f6177dd6a256f75c52c2dd7994e18d1b2: Status 404 returned error can't find the container with id f924d2912ee8a5e57c1e2bbc84abfb2f6177dd6a256f75c52c2dd7994e18d1b2 Jul 18 02:44:18.516598 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:18.516565 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vl8f/must-gather-gnbkn" event={"ID":"162b89fb-f83e-4cb1-945e-41f67b39d01b","Type":"ContainerStarted","Data":"f924d2912ee8a5e57c1e2bbc84abfb2f6177dd6a256f75c52c2dd7994e18d1b2"} Jul 18 02:44:19.524145 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:19.524103 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vl8f/must-gather-gnbkn" event={"ID":"162b89fb-f83e-4cb1-945e-41f67b39d01b","Type":"ContainerStarted","Data":"d2142a4914ff5c3e07a4b003f65f7669163878c36a0a0fad53800fb8ef63ef77"} Jul 18 02:44:19.524145 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:19.524157 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vl8f/must-gather-gnbkn" event={"ID":"162b89fb-f83e-4cb1-945e-41f67b39d01b","Type":"ContainerStarted","Data":"f9b6198eff066b10631e41425c8ad7b61c1cb09b1525741fd0acc1e59c6f59fc"} Jul 18 02:44:19.541243 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:19.541178 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6vl8f/must-gather-gnbkn" podStartSLOduration=1.689988718 podStartE2EDuration="2.541157107s" totalImagesPullingTime="851.168389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:44:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:19.539175772 +0000 UTC m=+2847.327512356" watchObservedRunningTime="2026-07-18 02:44:19.541157107 +0000 UTC m=+2847.329493690" Jul 18 02:44:20.505206 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:20.505172 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ljk5t_5c41bb40-6562-4a53-8641-8cb1b0a2c4f3/global-pull-secret-syncer/0.log" Jul 18 02:44:20.620734 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:20.620706 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9fbkh_f1827c91-f990-469b-b0df-d612d539262c/konnectivity-agent/0.log" Jul 18 02:44:20.716112 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:20.716079 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-187.ec2.internal_e85115e7389ad24d665892c86f4108f1/haproxy/0.log" Jul 18 02:44:25.299789 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.299755 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h_de6109e1-4420-4de5-bff3-d3f957245170/extract/0.log" Jul 18 02:44:25.338649 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.338567 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h_de6109e1-4420-4de5-bff3-d3f957245170/util/0.log" Jul 18 02:44:25.365552 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.365524 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dg78h_de6109e1-4420-4de5-bff3-d3f957245170/pull/0.log" Jul 18 02:44:25.396368 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.396340 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf_18429789-5590-4420-99cb-ada89afc4685/extract/0.log" Jul 18 02:44:25.419320 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.419289 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf_18429789-5590-4420-99cb-ada89afc4685/util/0.log" Jul 18 02:44:25.442276 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.442246 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e058fxf_18429789-5590-4420-99cb-ada89afc4685/pull/0.log" Jul 18 02:44:25.468717 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.468684 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm_b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22/extract/0.log" Jul 18 02:44:25.488930 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.488902 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm_b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22/util/0.log" Jul 18 02:44:25.522340 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.522305 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738jdcm_b5a77011-e9f3-453b-8fb1-1d3bc2c1ca22/pull/0.log" Jul 18 02:44:25.548324 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.548292 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc_ccbd3263-6251-4e1c-8fe7-713e2fc39d53/extract/0.log" Jul 18 02:44:25.576641 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.576536 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc_ccbd3263-6251-4e1c-8fe7-713e2fc39d53/util/0.log" Jul 18 02:44:25.610634 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:25.610605 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16rzwc_ccbd3263-6251-4e1c-8fe7-713e2fc39d53/pull/0.log" Jul 18 02:44:26.207868 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:26.207191 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-568fcc8d45-5r86j_b0269a7b-591d-4c64-889c-1dcb1660eadf/authorino/0.log" Jul 18 02:44:26.245414 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:26.244702 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zkb2x_04dd269b-879b-4b6a-9ed8-6959092e82ee/manager/0.log" Jul 18 02:44:26.362417 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:26.362376 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-26s7w_2b249822-89e4-46af-93d4-370a6935bc53/registry-server/0.log" Jul 18 02:44:26.639994 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:26.639299 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9sntf_394487e0-7e76-4cd7-837b-c31265df51c4/manager/0.log" Jul 18 02:44:28.820190 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:28.820154 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ws4ng_412b7285-a7b6-4388-85e2-9c2c337245e4/node-exporter/0.log" Jul 18 02:44:28.850031 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:28.850000 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ws4ng_412b7285-a7b6-4388-85e2-9c2c337245e4/kube-rbac-proxy/0.log" Jul 18 02:44:28.874627 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:28.874596 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ws4ng_412b7285-a7b6-4388-85e2-9c2c337245e4/init-textfile/0.log" Jul 18 02:44:30.331836 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.331803 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985"] Jul 18 02:44:30.337687 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.337660 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.340547 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.340520 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6vl8f\"/\"default-dockercfg-hrtqd\"" Jul 18 02:44:30.346332 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.346294 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985"] Jul 18 02:44:30.443572 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.443244 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-sys\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.443572 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.443302 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-lib-modules\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.443572 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.443338 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkczl\" (UniqueName: \"kubernetes.io/projected/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-kube-api-access-fkczl\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.443572 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.443381 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-podres\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.443572 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.443447 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-proc\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.544909 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.544877 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-sys\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.545135 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.544920 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-lib-modules\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.545135 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.544949 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkczl\" (UniqueName: \"kubernetes.io/projected/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-kube-api-access-fkczl\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.545135 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.545016 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-sys\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.545135 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.545068 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-podres\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.545135 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.545126 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-lib-modules\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.545408 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.545159 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-proc\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.545408 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.545284 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-proc\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.545408 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.545289 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-podres\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.554680 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.554656 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkczl\" (UniqueName: \"kubernetes.io/projected/0b027ef0-6544-4c1d-a6ff-724b5003c6c1-kube-api-access-fkczl\") pod \"perf-node-gather-daemonset-66985\" (UID: \"0b027ef0-6544-4c1d-a6ff-724b5003c6c1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.651136 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.651103 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:30.837735 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:30.837702 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985"] Jul 18 02:44:30.840815 ip-10-0-130-187 kubenswrapper[2573]: W0718 02:44:30.840786 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0b027ef0_6544_4c1d_a6ff_724b5003c6c1.slice/crio-493627e5061c1cf6547ce297a218277699521f50d3d9c21c9b954330a9bb3428 WatchSource:0}: Error finding container 493627e5061c1cf6547ce297a218277699521f50d3d9c21c9b954330a9bb3428: Status 404 returned error can't find the container with id 493627e5061c1cf6547ce297a218277699521f50d3d9c21c9b954330a9bb3428 Jul 18 02:44:31.653522 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:31.653468 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" event={"ID":"0b027ef0-6544-4c1d-a6ff-724b5003c6c1","Type":"ContainerStarted","Data":"6898a6a603f15cfeb10571152d2c9ace11ea52c7fdf110932aad9f9a893bda77"} Jul 18 02:44:31.654019 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:31.653562 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" event={"ID":"0b027ef0-6544-4c1d-a6ff-724b5003c6c1","Type":"ContainerStarted","Data":"493627e5061c1cf6547ce297a218277699521f50d3d9c21c9b954330a9bb3428"} Jul 18 02:44:31.654019 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:31.653620 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:31.673074 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:31.673021 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" podStartSLOduration=1.673003427 podStartE2EDuration="1.673003427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:44:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:31.672658824 +0000 UTC m=+2859.460995407" watchObservedRunningTime="2026-07-18 02:44:31.673003427 +0000 UTC m=+2859.461340008" Jul 18 02:44:31.690854 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:31.690820 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8669ff4d45-chxdr_69c85dc1-0740-4344-9f15-518f1210a748/console/0.log" Jul 18 02:44:31.736036 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:31.736010 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-fqsds_172a9bfd-1995-4d98-847f-a37dc04b2569/download-server/0.log" Jul 18 02:44:33.428492 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:33.428454 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q6gs9_c0625e69-2b51-4675-9080-fc196d8d8070/dns/0.log" Jul 18 02:44:33.449386 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:33.449360 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q6gs9_c0625e69-2b51-4675-9080-fc196d8d8070/kube-rbac-proxy/0.log" Jul 18 02:44:33.512601 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:33.512568 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fpf6q_4c6efe1e-41a0-429b-a45e-678499ab8f80/dns-node-resolver/0.log" Jul 18 02:44:34.079860 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:34.079825 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c544d7b4f-8b8n7_60d356df-e2e2-4536-8c3f-57b66a5bf040/registry/0.log" Jul 18 02:44:34.149008 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:34.148952 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m2vjh_3f6ca5e1-acf6-4e50-a04a-a0d039d44da9/node-ca/0.log" Jul 18 02:44:35.041509 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:35.041482 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4p724d_9e70ad11-ef7b-49b0-aafd-224381739a45/istio-proxy/0.log" Jul 18 02:44:35.236815 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:35.236786 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zt886_33381711-7e71-4b61-bf6a-e101f47a717b/istio-proxy/0.log" Jul 18 02:44:35.296562 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:35.296476 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zt886_33381711-7e71-4b61-bf6a-e101f47a717b/istio-proxy/1.log" Jul 18 02:44:35.368213 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:35.368176 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-986hh_be269f87-e314-4ca5-bcf4-24658f1a4dee/payload-processing/0.log" Jul 18 02:44:36.051400 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:36.051370 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j9bdq_25808b10-82d5-4db1-867a-7791b405f163/serve-healthcheck-canary/0.log" Jul 18 02:44:36.710941 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:36.710915 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zjfnx_d1abbff1-f365-4771-a560-10fee79a4ea2/kube-rbac-proxy/0.log" Jul 18 02:44:36.731749 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:36.731723 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zjfnx_d1abbff1-f365-4771-a560-10fee79a4ea2/exporter/0.log" Jul 18 02:44:36.755897 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:36.755853 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zjfnx_d1abbff1-f365-4771-a560-10fee79a4ea2/extractor/0.log" Jul 18 02:44:37.670299 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:37.670274 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-66985" Jul 18 02:44:39.404160 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:39.404132 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-hlgh6_081cb735-ba0b-4e64-910a-bb7e7ad6d55a/manager/0.log" Jul 18 02:44:39.428041 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:39.428014 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-hlgh6_081cb735-ba0b-4e64-910a-bb7e7ad6d55a/copy-manifests/0.log" Jul 18 02:44:40.573720 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:40.573675 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4p724d_9e70ad11-ef7b-49b0-aafd-224381739a45/istio-proxy/0.log" Jul 18 02:44:40.736203 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:40.736157 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zt886_33381711-7e71-4b61-bf6a-e101f47a717b/istio-proxy/0.log" Jul 18 02:44:40.801582 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:40.801536 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zt886_33381711-7e71-4b61-bf6a-e101f47a717b/istio-proxy/1.log" Jul 18 02:44:40.845375 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:40.845338 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-986hh_be269f87-e314-4ca5-bcf4-24658f1a4dee/payload-processing/0.log" Jul 18 02:44:40.863017 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:40.862985 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b64b4b7d4-5zdw7_1973b2b2-f1b1-48b4-9651-a7f652db1dc2/manager/0.log" Jul 18 02:44:42.029677 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:42.029597 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4p724d_9e70ad11-ef7b-49b0-aafd-224381739a45/istio-proxy/0.log" Jul 18 02:44:42.204786 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:42.204752 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zt886_33381711-7e71-4b61-bf6a-e101f47a717b/istio-proxy/0.log" Jul 18 02:44:42.266934 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:42.266899 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zt886_33381711-7e71-4b61-bf6a-e101f47a717b/istio-proxy/1.log" Jul 18 02:44:42.324914 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:42.324839 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-986hh_be269f87-e314-4ca5-bcf4-24658f1a4dee/payload-processing/0.log" Jul 18 02:44:46.221184 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:46.221106 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-2xgqt_d797431d-6fc4-47b6-8eca-5eeb7d664185/migrator/0.log" Jul 18 02:44:46.243404 ip-10-0-130-187 kubenswrapper[2573]: I0718 02:44:46.243379 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-2xgqt_d797431d-6fc4-47b6-8eca-5eeb7d664185/graceful-termination/0.log"