Jul 06 23:46:52.616719 ip-10-0-130-213 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 23:46:52.616731 ip-10-0-130-213 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 23:46:52.616738 ip-10-0-130-213 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 23:46:52.616963 ip-10-0-130-213 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 23:47:02.644641 ip-10-0-130-213 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 23:47:02.644659 ip-10-0-130-213 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 44eef0de94f346ecb65ad77e646a3b6d -- Jul 06 23:48:39.357204 ip-10-0-130-213 systemd[1]: Starting Kubernetes Kubelet... Jul 06 23:48:39.835990 ip-10-0-130-213 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:39.835990 ip-10-0-130-213 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 23:48:39.835990 ip-10-0-130-213 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:39.835990 ip-10-0-130-213 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 23:48:39.835990 ip-10-0-130-213 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:39.836941 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.836844 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 23:48:39.843249 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843219 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:39.843249 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843243 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:39.843249 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843250 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:39.843249 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843255 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843259 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843264 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843268 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843272 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843275 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843279 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843283 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843286 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843289 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843293 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843296 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843299 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843303 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843307 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843311 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843314 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843317 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843321 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843348 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:39.843536 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843353 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843357 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843362 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843367 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843371 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843375 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843382 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843387 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843392 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843396 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843400 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843404 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843408 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843412 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843418 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843423 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843427 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843431 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843436 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:39.844371 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843440 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843444 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843448 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843454 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843458 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843462 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843466 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843470 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843476 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843480 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843484 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843488 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843492 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843496 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843500 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843505 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843509 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843513 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843517 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843521 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:39.845197 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843525 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843529 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843533 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843537 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843541 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843547 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843555 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843560 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843565 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843569 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843574 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843580 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843584 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843588 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843592 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843596 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843600 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843604 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843609 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:39.846060 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843613 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843619 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843626 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843631 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843635 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843639 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843643 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843647 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843651 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843655 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843660 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843664 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843668 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843672 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843676 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843680 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843684 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.843688 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.844972 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.844985 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:39.846883 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.844991 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.844997 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845001 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845006 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845010 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845015 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845021 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845027 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845031 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845035 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845040 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845044 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845048 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845052 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845056 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845062 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845066 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845070 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845074 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:39.847750 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845079 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845083 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845087 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845092 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845097 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845101 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845105 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845112 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845118 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845124 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845128 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845132 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845137 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845142 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845146 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845150 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845154 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845159 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845164 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845168 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:39.848436 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845172 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845176 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845181 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845185 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845189 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845193 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845197 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845201 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845205 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845210 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845215 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845221 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845226 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845231 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845238 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845243 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845248 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845252 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845258 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:39.848983 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845262 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845267 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845272 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845276 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845282 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845287 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845292 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845296 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845301 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845305 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845309 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845313 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845317 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845322 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845326 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845331 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845335 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845340 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845344 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:39.849507 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845348 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845352 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845356 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845360 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845366 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845371 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845375 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845380 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845383 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845388 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845392 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845396 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845400 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845404 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845408 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845412 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845416 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845420 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845424 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.845429 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:39.850221 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846085 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846093 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846098 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846104 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846109 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846113 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846117 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846121 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846126 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846130 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846135 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846139 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846144 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846148 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846152 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846156 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846164 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846168 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846173 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846177 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:39.850711 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846181 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846186 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846190 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846194 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846198 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846202 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846206 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846210 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846214 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846218 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846223 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846227 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846231 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846235 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846240 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846255 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846261 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846266 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846271 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:39.851316 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846276 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846280 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846285 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846289 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846293 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846297 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846301 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846305 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846309 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846315 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846320 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846325 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846329 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846333 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846337 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846341 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846345 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846349 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846353 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:39.851817 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846356 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846363 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846368 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846373 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846378 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846383 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846388 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846392 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846397 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846402 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846406 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846410 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846415 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846419 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846423 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846427 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846431 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846436 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846440 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846445 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:39.852307 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846451 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846455 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846461 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846465 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846470 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846474 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846480 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846485 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846491 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846496 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846500 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846504 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846508 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846513 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846517 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846521 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846525 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846529 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846533 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846537 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:39.852783 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.846541 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846651 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846662 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846672 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846679 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846686 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846691 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846697 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846704 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846709 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846713 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846718 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846724 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846729 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846736 2558 flags.go:64] FLAG: --cgroup-root="" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846740 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846744 2558 flags.go:64] FLAG: --client-ca-file="" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846750 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846755 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846762 2558 flags.go:64] FLAG: --cluster-domain="" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846768 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846773 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846778 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846783 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 23:48:39.853317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846789 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846794 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846799 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846804 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846809 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846813 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846818 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846822 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846827 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846833 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846838 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846843 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846866 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846871 2558 flags.go:64] FLAG: --enable-server="true" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846876 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846883 2558 flags.go:64] FLAG: --event-burst="100" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846888 2558 flags.go:64] FLAG: --event-qps="50" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846892 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846897 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846901 2558 flags.go:64] FLAG: --eviction-hard="" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846907 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846911 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846918 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846922 2558 flags.go:64] FLAG: --eviction-soft="" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846927 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 23:48:39.853928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846932 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846938 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846943 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846949 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846953 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846958 2558 flags.go:64] FLAG: --feature-gates="" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846964 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846969 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846973 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846978 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846983 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846988 2558 flags.go:64] FLAG: --help="false" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.846993 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-130-213.ec2.internal" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847006 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847010 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847015 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847020 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847026 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847030 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847035 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847039 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847044 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847048 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847053 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 23:48:39.854607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847058 2558 flags.go:64] FLAG: --kube-reserved="" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847062 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847066 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847071 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847076 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847082 2558 flags.go:64] FLAG: --lock-file="" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847087 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847091 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847096 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847105 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847110 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847117 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847121 2558 flags.go:64] FLAG: --logging-format="text" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847126 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847131 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847136 2558 flags.go:64] FLAG: --manifest-url="" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847140 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847147 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847152 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847158 2558 flags.go:64] FLAG: --max-pods="110" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847163 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847167 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847171 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847176 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847180 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 23:48:39.855315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847185 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847189 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847201 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847206 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847211 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847215 2558 flags.go:64] FLAG: --pod-cidr="" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847219 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847228 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847232 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847237 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847241 2558 flags.go:64] FLAG: --port="10250" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847246 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847250 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-087dee0d121b775fc" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847257 2558 flags.go:64] FLAG: --qos-reserved="" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847262 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847266 2558 flags.go:64] FLAG: --register-node="true" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847271 2558 flags.go:64] FLAG: --register-with-taints="" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847277 2558 flags.go:64] FLAG: --registry-burst="10" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847284 2558 flags.go:64] FLAG: --registry-qps="5" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847289 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847293 2558 flags.go:64] FLAG: --reserved-memory="" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847299 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847303 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847308 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847312 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 23:48:39.855953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847317 2558 flags.go:64] FLAG: --runonce="false" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847321 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847326 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847331 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847335 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847340 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847344 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847349 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847354 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847358 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847363 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847367 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847371 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847376 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847380 2558 flags.go:64] FLAG: --system-cgroups="" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847385 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847394 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847398 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847403 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847418 2558 flags.go:64] FLAG: --tls-min-version="" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847424 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847429 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847434 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847438 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847444 2558 flags.go:64] FLAG: --v="2" Jul 06 23:48:39.856563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847454 2558 flags.go:64] FLAG: --version="false" Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847465 2558 flags.go:64] FLAG: --vmodule="" Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847471 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.847477 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847678 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847684 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847690 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847694 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847699 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847704 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847708 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847715 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847720 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847727 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847732 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847737 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847742 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847746 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847750 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847754 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:39.857228 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847758 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847761 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847765 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847769 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847773 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847777 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847781 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847787 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847792 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847796 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847800 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847805 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847811 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847815 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847823 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847828 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847833 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847837 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847841 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847863 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:39.857712 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847868 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847872 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847877 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847881 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847885 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847889 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847893 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847896 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847901 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847905 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847910 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847914 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847917 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847921 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847926 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847930 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847934 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847938 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847942 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847947 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:39.858277 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847951 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847956 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847960 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847964 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847970 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847974 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847978 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847984 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847989 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847993 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.847997 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848001 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848006 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848010 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848014 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848020 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848025 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848029 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848033 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:39.858768 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848038 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848042 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848046 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848050 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848054 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848059 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848063 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848067 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848071 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848075 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848079 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848083 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848089 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848093 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848097 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848101 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848105 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848111 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848115 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848119 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:39.859258 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848123 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848129 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848133 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.848137 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.848924 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.855863 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.855879 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.855899 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.855906 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855962 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855966 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855970 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855973 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855976 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:39.859754 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855979 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855981 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855984 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855986 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855989 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855992 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855994 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.855997 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856000 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856003 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856005 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856008 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856010 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856013 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856015 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856018 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856020 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856023 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856026 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856030 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:39.860131 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856033 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856035 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856038 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856040 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856045 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856048 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856050 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856053 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856055 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856058 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856060 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856063 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856065 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856068 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856070 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856073 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856076 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856078 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856081 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856083 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:39.860616 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856085 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856088 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856092 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856095 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856098 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856101 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856104 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856106 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856108 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856111 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856113 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856116 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856118 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856121 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856123 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856126 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856129 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856135 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856138 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:39.861122 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856140 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856142 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856145 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856148 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856152 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856155 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856158 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856160 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856163 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856166 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856169 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856172 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856174 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856177 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856180 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856183 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856185 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856188 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856190 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856193 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:39.861593 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856195 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856198 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856200 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856203 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856205 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856208 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856210 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856213 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856216 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856219 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856221 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856224 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856226 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856229 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:39.862222 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856231 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.856236 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856351 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856356 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856359 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856362 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856364 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856367 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856370 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856373 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856375 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856378 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856381 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856383 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856386 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:39.862559 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856389 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856391 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856394 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856396 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856399 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856401 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856404 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856406 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856409 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856411 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856414 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856416 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856419 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856421 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856424 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856427 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856429 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856431 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856434 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856436 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:39.862948 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856439 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856441 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856444 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856446 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856448 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856451 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856454 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856456 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856459 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856461 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856463 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856466 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856468 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856471 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856474 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856476 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856479 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856482 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856486 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:39.863435 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856488 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856491 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856493 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856496 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856498 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856501 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856503 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856506 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856508 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856511 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856513 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856515 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856518 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856521 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856523 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856525 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856528 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856530 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856533 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856535 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:39.863923 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856537 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856540 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856542 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856545 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856547 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856550 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856552 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856554 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856557 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856559 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856561 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856564 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856566 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856569 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856571 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856574 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856577 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856579 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856582 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:39.864405 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856586 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856589 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856593 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856597 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856599 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856602 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856604 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:39.856606 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.856611 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.856720 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.862646 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.864054 2558 server.go:1019] "Starting client certificate rotation" Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.864147 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:48:39.864878 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.864183 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:48:39.867343 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.867329 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:48:39.872172 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.872148 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:48:39.887357 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.887330 2558 log.go:25] "Validated CRI v1 runtime API" Jul 06 23:48:39.889420 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.889398 2558 log.go:25] "Validated CRI v1 image API" Jul 06 23:48:39.890321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.890305 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 23:48:39.894040 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.894024 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:48:39.894903 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.894876 2558 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8f0d87dd-958b-4a12-a438-d7112979becc:/dev/nvme0n1p3 b71c0394-7529-48c0-baf6-7853dfed8e22:/dev/nvme0n1p4] Jul 06 23:48:39.894960 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.894902 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 23:48:39.899292 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.899179 2558 manager.go:216] Machine: {Timestamp:2026-07-06 23:48:39.898649177 +0000 UTC m=+0.408810125 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100255 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d91bc3cdd8860f26bc38b37aa32ae SystemUUID:ec2d91bc-3cdd-8860-f26b-c38b37aa32ae BootID:44eef0de-94f3-46ec-b65a-d77e646a3b6d Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:da:be:2c:0b:47 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:da:be:2c:0b:47 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:e5:f9:96:5d:7c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 23:48:39.899292 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.899285 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 23:48:39.899416 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.899367 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 23:48:39.903176 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.903146 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 23:48:39.903330 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.903179 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-213.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 23:48:39.904308 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.904295 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 23:48:39.904346 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.904311 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 23:48:39.904346 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.904319 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:48:39.904346 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.904333 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:48:39.905777 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.905764 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 23:48:39.909575 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.909558 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:48:39.909714 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.909704 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 23:48:39.909746 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.909737 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 06 23:48:39.909788 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.909747 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 23:48:39.909788 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.909766 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 23:48:39.909788 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.909780 2558 kubelet.go:398] "Adding apiserver pod source" Jul 06 23:48:39.909948 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.909793 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 23:48:39.910322 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.910294 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:48:39.910372 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.910333 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:48:39.912585 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.912563 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 23:48:39.914519 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.914505 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 23:48:39.914574 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.914537 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 23:48:39.915722 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915710 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 23:48:39.915722 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915725 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915731 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915743 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915751 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915756 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915764 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915769 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915775 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915781 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915789 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915795 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915818 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 23:48:39.915834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.915837 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 23:48:39.919430 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.919410 2558 server.go:1268] "Started kubelet" Jul 06 23:48:39.919686 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.919657 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 23:48:39.920506 ip-10-0-130-213 systemd[1]: Started Kubernetes Kubelet. Jul 06 23:48:39.923599 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.923390 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 23:48:39.923726 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.923633 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 23:48:39.924472 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.924434 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 23:48:39.924766 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.924735 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 23:48:39.924833 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.924789 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 23:48:39.925771 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.925644 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 23:48:39.925916 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.925897 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 23:48:39.926439 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.926407 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 06 23:48:39.932643 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.932613 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 23:48:39.932643 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.932641 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 23:48:39.932843 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.932733 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-213.ec2.internal\" not found" Jul 06 23:48:39.933995 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.933979 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 23:48:39.935167 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.935143 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 23:48:39.935167 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.935170 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 23:48:39.935373 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.935361 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 06 23:48:39.935373 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.935373 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 06 23:48:39.935515 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.935496 2558 factory.go:55] Registering systemd factory Jul 06 23:48:39.935566 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.935520 2558 factory.go:223] Registration of the systemd container factory successfully Jul 06 23:48:39.935663 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.935642 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 23:48:39.935750 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.935721 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 23:48:39.936430 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.936413 2558 factory.go:153] Registering CRI-O factory Jul 06 23:48:39.936430 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.936431 2558 factory.go:223] Registration of the crio container factory successfully Jul 06 23:48:39.936564 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.936519 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 23:48:39.936564 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.936557 2558 factory.go:103] Registering Raw factory Jul 06 23:48:39.936658 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.936568 2558 manager.go:1196] Started watching for new ooms in manager Jul 06 23:48:39.937049 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.937033 2558 manager.go:349] Starting recovery of all containers Jul 06 23:48:39.937996 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.936074 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-213.ec2.internal.18bfd839bab84c2c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-213.ec2.internal,UID:ip-10-0-130-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-213.ec2.internal,},FirstTimestamp:2026-07-06 23:48:39.919381548 +0000 UTC m=+0.429542497,LastTimestamp:2026-07-06 23:48:39.919381548 +0000 UTC m=+0.429542497,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-213.ec2.internal,}" Jul 06 23:48:39.942315 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.942133 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 23:48:39.947639 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.947616 2558 manager.go:354] Recovery completed Jul 06 23:48:39.952056 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.952041 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:39.954186 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.954159 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:39.954278 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.954202 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:39.954278 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.954214 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:39.954756 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.954742 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 23:48:39.954799 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.954756 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 23:48:39.954799 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.954774 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:48:39.954799 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.954787 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:39.955658 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.955644 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:39.955720 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.955683 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:39.955720 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.955693 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:39.956053 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.956042 2558 policy_none.go:49] "None policy: Start" Jul 06 23:48:39.956095 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.956058 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 23:48:39.956095 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.956068 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 23:48:39.956936 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.956926 2558 policy_none.go:47] "Start" Jul 06 23:48:39.957776 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.957687 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-213.ec2.internal.18bfd839bccb5eed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-213.ec2.internal,UID:ip-10-0-130-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-213.ec2.internal,},FirstTimestamp:2026-07-06 23:48:39.954185965 +0000 UTC m=+0.464346914,LastTimestamp:2026-07-06 23:48:39.954185965 +0000 UTC m=+0.464346914,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-213.ec2.internal,}" Jul 06 23:48:39.964119 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.964092 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xslp8" Jul 06 23:48:39.968821 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.968744 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-213.ec2.internal.18bfd839bccbb51b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-213.ec2.internal,UID:ip-10-0-130-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-213.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-213.ec2.internal,},FirstTimestamp:2026-07-06 23:48:39.954208027 +0000 UTC m=+0.464368975,LastTimestamp:2026-07-06 23:48:39.954208027 +0000 UTC m=+0.464368975,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-213.ec2.internal,}" Jul 06 23:48:39.973471 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.973386 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-213.ec2.internal.18bfd839bccbdd74 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-213.ec2.internal,UID:ip-10-0-130-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-213.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-213.ec2.internal,},FirstTimestamp:2026-07-06 23:48:39.954218356 +0000 UTC m=+0.464379304,LastTimestamp:2026-07-06 23:48:39.954218356 +0000 UTC m=+0.464379304,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-213.ec2.internal,}" Jul 06 23:48:39.979057 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.979035 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xslp8" Jul 06 23:48:39.993456 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.993440 2558 manager.go:337] "Starting Device Plugin manager" Jul 06 23:48:40.017036 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.993471 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 23:48:40.017036 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.993481 2558 server.go:87] "Starting device plugin registration server" Jul 06 23:48:40.017036 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.993724 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 23:48:40.017036 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.993735 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 23:48:40.017036 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.993892 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 23:48:40.017036 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.993982 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 23:48:40.017036 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:39.993991 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 23:48:40.017036 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.994629 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 23:48:40.017036 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:39.994681 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-213.ec2.internal\" not found" Jul 06 23:48:40.044954 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.044922 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 23:48:40.045110 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.044967 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 23:48:40.045110 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.044991 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 23:48:40.045110 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.045023 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 23:48:40.047705 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.047686 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:40.094753 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.094678 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:40.095897 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.095878 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:40.096019 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.095913 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:40.096019 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.095923 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:40.096019 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.095950 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.103984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.103968 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.104085 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.103989 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-213.ec2.internal\": node \"ip-10-0-130-213.ec2.internal\" not found" Jul 06 23:48:40.142286 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.142260 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-213.ec2.internal\" not found" Jul 06 23:48:40.145428 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.145406 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal"] Jul 06 23:48:40.145484 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.145474 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:40.146456 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.146432 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:40.146583 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.146473 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:40.146583 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.146487 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:40.148684 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.148669 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:40.148844 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.148830 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.148939 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.148884 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:40.149455 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.149441 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:40.149533 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.149461 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:40.149533 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.149474 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:40.149533 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.149492 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:40.149533 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.149512 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:40.149533 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.149521 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:40.151637 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.151623 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.151689 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.151648 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:40.152356 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.152337 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:40.152438 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.152367 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:40.152438 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.152380 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:40.175714 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.175689 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-213.ec2.internal\" not found" node="ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.180176 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.180160 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-213.ec2.internal\" not found" node="ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.237419 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.237384 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9354effc7df6757b13f443e5e88f79e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal\" (UID: \"a9354effc7df6757b13f443e5e88f79e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.237419 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.237419 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9354effc7df6757b13f443e5e88f79e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal\" (UID: \"a9354effc7df6757b13f443e5e88f79e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.237585 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.237438 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/373575230b88bb4ec2f11847d42c51bd-config\") pod \"kube-apiserver-proxy-ip-10-0-130-213.ec2.internal\" (UID: \"373575230b88bb4ec2f11847d42c51bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.242548 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.242526 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-213.ec2.internal\" not found" Jul 06 23:48:40.338490 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.338458 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9354effc7df6757b13f443e5e88f79e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal\" (UID: \"a9354effc7df6757b13f443e5e88f79e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.338490 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.338477 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9354effc7df6757b13f443e5e88f79e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal\" (UID: \"a9354effc7df6757b13f443e5e88f79e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.338568 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.338509 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9354effc7df6757b13f443e5e88f79e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal\" (UID: \"a9354effc7df6757b13f443e5e88f79e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.338568 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.338539 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/373575230b88bb4ec2f11847d42c51bd-config\") pod \"kube-apiserver-proxy-ip-10-0-130-213.ec2.internal\" (UID: \"373575230b88bb4ec2f11847d42c51bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.338568 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.338553 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9354effc7df6757b13f443e5e88f79e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal\" (UID: \"a9354effc7df6757b13f443e5e88f79e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.338656 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.338568 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/373575230b88bb4ec2f11847d42c51bd-config\") pod \"kube-apiserver-proxy-ip-10-0-130-213.ec2.internal\" (UID: \"373575230b88bb4ec2f11847d42c51bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.343559 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.343539 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-213.ec2.internal\" not found" Jul 06 23:48:40.444539 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.444465 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-213.ec2.internal\" not found" Jul 06 23:48:40.478661 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.478633 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.485760 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.485582 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.544587 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.544544 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-213.ec2.internal\" not found" Jul 06 23:48:40.645152 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.645117 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-213.ec2.internal\" not found" Jul 06 23:48:40.745940 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.745843 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-213.ec2.internal\" not found" Jul 06 23:48:40.746490 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.746475 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:40.833748 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.833712 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.849570 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.849544 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:48:40.852924 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.852897 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal" Jul 06 23:48:40.862662 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.862642 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:48:40.863693 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.863677 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 23:48:40.863816 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.863790 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:48:40.863874 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.863824 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:48:40.909958 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.909920 2558 apiserver.go:52] "Watching apiserver" Jul 06 23:48:40.928645 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.928615 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 23:48:40.931043 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.930979 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-c5zp8","openshift-network-diagnostics/network-check-target-zdbst","kube-system/global-pull-secret-syncer-8dl25","kube-system/konnectivity-agent-2nd7r","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr","openshift-image-registry/node-ca-brsvk","openshift-network-operator/iptables-alerter-z6dqd","openshift-ovn-kubernetes/ovnkube-node-2z7v2","kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal","openshift-cluster-node-tuning-operator/tuned-hs2xd","openshift-dns/node-resolver-wxmxp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal","openshift-multus/multus-24nhh","openshift-multus/multus-additional-cni-plugins-tw7t9"] Jul 06 23:48:40.933111 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.933090 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 23:48:40.933576 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.933558 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:40.937372 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.937329 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.938176 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.938159 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 23:48:40.938256 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.938230 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.938787 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.938773 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.938825 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.938813 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ck7zt\"" Jul 06 23:48:40.939438 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.939422 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:40.939530 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.939483 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:48:40.941359 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941333 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-cni-dir\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941444 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941382 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-cnibin\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941444 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941424 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-run-multus-certs\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941544 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941449 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnvd5\" (UniqueName: \"kubernetes.io/projected/a58005e6-88fa-4cda-bf75-bff326b46b7c-kube-api-access-wnvd5\") pod \"iptables-alerter-z6dqd\" (UID: \"a58005e6-88fa-4cda-bf75-bff326b46b7c\") " pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:40.941544 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941464 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/847d0bbc-bde0-4006-ad49-b87007f8923c-cni-binary-copy\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941544 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941479 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-socket-dir-parent\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941544 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941502 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-run-netns\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941544 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941522 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-var-lib-kubelet\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941772 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941557 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-etc-kubernetes\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941772 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941594 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-system-cni-dir\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941772 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941618 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-os-release\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941772 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941643 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-var-lib-cni-bin\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941772 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941667 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-hostroot\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941772 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941688 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-conf-dir\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.941772 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941712 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a58005e6-88fa-4cda-bf75-bff326b46b7c-iptables-alerter-script\") pod \"iptables-alerter-z6dqd\" (UID: \"a58005e6-88fa-4cda-bf75-bff326b46b7c\") " pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:40.941772 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941737 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a58005e6-88fa-4cda-bf75-bff326b46b7c-host-slash\") pod \"iptables-alerter-z6dqd\" (UID: \"a58005e6-88fa-4cda-bf75-bff326b46b7c\") " pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:40.941772 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941774 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:48:40.942139 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941780 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-run-k8s-cni-cncf-io\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.942139 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941816 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-var-lib-cni-multus\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.942139 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941840 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-daemon-config\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.942139 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.941896 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tskc\" (UniqueName: \"kubernetes.io/projected/847d0bbc-bde0-4006-ad49-b87007f8923c-kube-api-access-4tskc\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:40.942396 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.942376 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.942468 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.942419 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.942795 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.942776 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 23:48:40.942948 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.942824 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dhrjm\"" Jul 06 23:48:40.942948 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.942825 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 23:48:40.944088 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.944072 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:40.944383 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.944298 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c7rnr\"" Jul 06 23:48:40.944811 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.944795 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 23:48:40.944920 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.944894 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 23:48:40.946386 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.946364 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:40.946793 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.946717 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.946877 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.946798 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pb266\"" Jul 06 23:48:40.947087 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.947065 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.947177 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.947111 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 23:48:40.948268 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.948245 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:48:40.948703 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.948691 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.949222 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.949205 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:40.949768 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.949750 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 23:48:40.949987 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.949864 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ndxkx\"" Jul 06 23:48:40.950579 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.950561 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.951420 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.951400 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:40.951505 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.951479 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:48:40.951750 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.951735 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l8mbt\"" Jul 06 23:48:40.952510 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.952490 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 23:48:40.952607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.952491 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 23:48:40.954102 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.954003 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:40.954102 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:40.954070 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:48:40.956055 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.956035 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:40.959378 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.959359 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:40.959515 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.959477 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.959655 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.959636 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hjz5k\"" Jul 06 23:48:40.959786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.959768 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.961922 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.961903 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:40.963321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.962839 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.963321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.963128 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bd996\"" Jul 06 23:48:40.963321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.963269 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.966572 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.966547 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 23:48:40.966708 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.966549 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 23:48:40.966840 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.966719 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.968182 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.968165 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 23:48:40.968291 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.968241 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.968291 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.968165 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t59k7\"" Jul 06 23:48:40.968515 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.968445 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 23:48:40.977448 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.977431 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7bpgb" Jul 06 23:48:40.981546 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.981514 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:43:40 +0000 UTC" deadline="2027-12-21 21:55:05.185463605 +0000 UTC" Jul 06 23:48:40.981603 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.981546 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12790h6m24.203920527s" Jul 06 23:48:40.992561 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:40.992540 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7bpgb" Jul 06 23:48:41.035911 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.035890 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 23:48:41.042387 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042356 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-device-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.042497 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042393 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ff9b33f-ec62-4d90-80dc-745c2ffa9498-serviceca\") pod \"node-ca-brsvk\" (UID: \"9ff9b33f-ec62-4d90-80dc-745c2ffa9498\") " pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:41.042497 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042411 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4ebb3c3f-eda3-4c28-987b-11627a176748-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.042497 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042437 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-run-netns\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.042497 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042469 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-var-lib-openvswitch\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.042497 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042489 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-run-ovn\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.042724 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042518 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-daemon-config\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.042724 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042532 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-run\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.042724 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042590 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-log-socket\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.042724 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042703 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-cni-bin\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.042936 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042749 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-socket-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.042936 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042779 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-cnibin\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.042936 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042805 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7g6b\" (UniqueName: \"kubernetes.io/projected/4ebb3c3f-eda3-4c28-987b-11627a176748-kube-api-access-w7g6b\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.042936 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042832 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-kubelet\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.042936 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042875 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnvd5\" (UniqueName: \"kubernetes.io/projected/a58005e6-88fa-4cda-bf75-bff326b46b7c-kube-api-access-wnvd5\") pod \"iptables-alerter-z6dqd\" (UID: \"a58005e6-88fa-4cda-bf75-bff326b46b7c\") " pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:41.042936 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042913 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-run-netns\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042941 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-var-lib-kubelet\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.042986 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043011 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-run-netns\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043018 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4ebb3c3f-eda3-4c28-987b-11627a176748-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043034 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-var-lib-kubelet\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043048 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4423aa72-95c7-4610-9d94-76b3f1c2b76e-tmp-dir\") pod \"node-resolver-wxmxp\" (UID: \"4423aa72-95c7-4610-9d94-76b3f1c2b76e\") " pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043085 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-system-cni-dir\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043134 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-system-cni-dir\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043144 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-var-lib-cni-bin\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043178 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-modprobe-d\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043204 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-var-lib-kubelet\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043183 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-var-lib-cni-bin\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043226 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09e10046-6938-48cf-9a41-9a726dbc472f-etc-tuned\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043262 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-os-release\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043262 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-daemon-config\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043283 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-run-openvswitch\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043307 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a58005e6-88fa-4cda-bf75-bff326b46b7c-iptables-alerter-script\") pod \"iptables-alerter-z6dqd\" (UID: \"a58005e6-88fa-4cda-bf75-bff326b46b7c\") " pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043331 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a58005e6-88fa-4cda-bf75-bff326b46b7c-host-slash\") pod \"iptables-alerter-z6dqd\" (UID: \"a58005e6-88fa-4cda-bf75-bff326b46b7c\") " pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043405 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-run-k8s-cni-cncf-io\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043421 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a58005e6-88fa-4cda-bf75-bff326b46b7c-host-slash\") pod \"iptables-alerter-z6dqd\" (UID: \"a58005e6-88fa-4cda-bf75-bff326b46b7c\") " pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043463 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-var-lib-cni-multus\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043493 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-run-k8s-cni-cncf-io\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043538 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-slash\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043568 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-run-ovn-kubernetes\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043606 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-cnibin\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043632 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-var-lib-cni-multus\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043669 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-run-multus-certs\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043693 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-run-systemd\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043731 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/847d0bbc-bde0-4006-ad49-b87007f8923c-cni-binary-copy\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.043808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043748 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-cnibin\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043767 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsr5s\" (UniqueName: \"kubernetes.io/projected/6c32c893-7163-4346-a810-ac3b1aed5eeb-kube-api-access-vsr5s\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043798 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-sysconfig\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043800 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-host-run-multus-certs\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043805 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a58005e6-88fa-4cda-bf75-bff326b46b7c-iptables-alerter-script\") pod \"iptables-alerter-z6dqd\" (UID: \"a58005e6-88fa-4cda-bf75-bff326b46b7c\") " pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043840 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09e10046-6938-48cf-9a41-9a726dbc472f-tmp\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043882 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/707c4468-3606-41ee-8a87-dc5764875fa5-ovn-node-metrics-cert\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043902 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-hostroot\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043921 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-systemd\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043940 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-system-cni-dir\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043970 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4ebb3c3f-eda3-4c28-987b-11627a176748-cni-binary-copy\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043987 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-hostroot\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.043998 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-systemd-units\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044033 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-cni-netd\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044058 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/707c4468-3606-41ee-8a87-dc5764875fa5-ovnkube-config\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044076 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/707c4468-3606-41ee-8a87-dc5764875fa5-ovnkube-script-lib\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044096 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-lib-modules\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.044598 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044113 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-dbus\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044129 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-cni-dir\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044149 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-sys\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044175 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-registration-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044203 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-cni-dir\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044226 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-etc-kubernetes\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044243 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ff9b33f-ec62-4d90-80dc-745c2ffa9498-host\") pod \"node-ca-brsvk\" (UID: \"9ff9b33f-ec62-4d90-80dc-745c2ffa9498\") " pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044284 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/707c4468-3606-41ee-8a87-dc5764875fa5-env-overrides\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044286 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-etc-kubernetes\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044322 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fb6985f2-1b61-47fd-a1b0-17429a573c7d-konnectivity-ca\") pod \"konnectivity-agent-2nd7r\" (UID: \"fb6985f2-1b61-47fd-a1b0-17429a573c7d\") " pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044349 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-conf-dir\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044368 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-etc-selinux\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044396 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-sys-fs\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044428 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-conf-dir\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044473 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkcsk\" (UniqueName: \"kubernetes.io/projected/9ff9b33f-ec62-4d90-80dc-745c2ffa9498-kube-api-access-dkcsk\") pod \"node-ca-brsvk\" (UID: \"9ff9b33f-ec62-4d90-80dc-745c2ffa9498\") " pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044522 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-host\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044559 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmkdp\" (UniqueName: \"kubernetes.io/projected/09e10046-6938-48cf-9a41-9a726dbc472f-kube-api-access-xmkdp\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.045296 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044616 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4423aa72-95c7-4610-9d94-76b3f1c2b76e-hosts-file\") pod \"node-resolver-wxmxp\" (UID: \"4423aa72-95c7-4610-9d94-76b3f1c2b76e\") " pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044642 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-etc-openvswitch\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044669 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tskc\" (UniqueName: \"kubernetes.io/projected/847d0bbc-bde0-4006-ad49-b87007f8923c-kube-api-access-4tskc\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044735 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044763 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km997\" (UniqueName: \"kubernetes.io/projected/4423aa72-95c7-4610-9d94-76b3f1c2b76e-kube-api-access-km997\") pod \"node-resolver-wxmxp\" (UID: \"4423aa72-95c7-4610-9d94-76b3f1c2b76e\") " pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044785 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-node-log\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044812 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044869 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq6j8\" (UniqueName: \"kubernetes.io/projected/707c4468-3606-41ee-8a87-dc5764875fa5-kube-api-access-fq6j8\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044906 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-kubelet-config\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044930 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fb6985f2-1b61-47fd-a1b0-17429a573c7d-agent-certs\") pod \"konnectivity-agent-2nd7r\" (UID: \"fb6985f2-1b61-47fd-a1b0-17429a573c7d\") " pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044963 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldkgr\" (UniqueName: \"kubernetes.io/projected/c5c374b5-74a1-41b6-826c-288b284cc627-kube-api-access-ldkgr\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044980 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/847d0bbc-bde0-4006-ad49-b87007f8923c-cni-binary-copy\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.044986 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbsxc\" (UniqueName: \"kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc\") pod \"network-check-target-zdbst\" (UID: \"7de4f318-5759-4f4f-870e-01b72f5f3e90\") " pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.045022 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-kubernetes\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.045054 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-sysctl-d\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.045107 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:41.045786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.045133 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-socket-dir-parent\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.046386 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.045157 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-sysctl-conf\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.046386 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.045181 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.046386 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.045218 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-multus-socket-dir-parent\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.046386 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.045250 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-os-release\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.046386 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.045308 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/847d0bbc-bde0-4006-ad49-b87007f8923c-os-release\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.046386 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:41.046049 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9354effc7df6757b13f443e5e88f79e.slice/crio-bfad68cae46d484489bbf3a8dc19b5eb2c8ee203e089cc85b458d96e4ddd9c41 WatchSource:0}: Error finding container bfad68cae46d484489bbf3a8dc19b5eb2c8ee203e089cc85b458d96e4ddd9c41: Status 404 returned error can't find the container with id bfad68cae46d484489bbf3a8dc19b5eb2c8ee203e089cc85b458d96e4ddd9c41 Jul 06 23:48:41.046386 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:41.046307 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod373575230b88bb4ec2f11847d42c51bd.slice/crio-85242e81115ae7ac54473434ab7399ee77b3b799cb48cba4d0bc1791b189a039 WatchSource:0}: Error finding container 85242e81115ae7ac54473434ab7399ee77b3b799cb48cba4d0bc1791b189a039: Status 404 returned error can't find the container with id 85242e81115ae7ac54473434ab7399ee77b3b799cb48cba4d0bc1791b189a039 Jul 06 23:48:41.055164 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.055145 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:48:41.062148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.062122 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 23:48:41.065404 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.065376 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnvd5\" (UniqueName: \"kubernetes.io/projected/a58005e6-88fa-4cda-bf75-bff326b46b7c-kube-api-access-wnvd5\") pod \"iptables-alerter-z6dqd\" (UID: \"a58005e6-88fa-4cda-bf75-bff326b46b7c\") " pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:41.067717 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.067689 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tskc\" (UniqueName: \"kubernetes.io/projected/847d0bbc-bde0-4006-ad49-b87007f8923c-kube-api-access-4tskc\") pod \"multus-24nhh\" (UID: \"847d0bbc-bde0-4006-ad49-b87007f8923c\") " pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.145666 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145625 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsr5s\" (UniqueName: \"kubernetes.io/projected/6c32c893-7163-4346-a810-ac3b1aed5eeb-kube-api-access-vsr5s\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.145666 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145663 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-sysconfig\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.145897 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145725 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-sysconfig\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.145897 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145767 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09e10046-6938-48cf-9a41-9a726dbc472f-tmp\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.145897 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145798 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/707c4468-3606-41ee-8a87-dc5764875fa5-ovn-node-metrics-cert\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.145897 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145816 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-systemd\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.145897 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145839 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-system-cni-dir\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.145897 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145881 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4ebb3c3f-eda3-4c28-987b-11627a176748-cni-binary-copy\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.146178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145896 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-systemd\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.146178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145905 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-systemd-units\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.146178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145928 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-cni-netd\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.146178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145933 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-system-cni-dir\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.146178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145951 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/707c4468-3606-41ee-8a87-dc5764875fa5-ovnkube-config\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.146178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145973 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-systemd-units\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.146178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145976 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/707c4468-3606-41ee-8a87-dc5764875fa5-ovnkube-script-lib\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.146178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.145998 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-lib-modules\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.146178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146006 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-cni-netd\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.146178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146052 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-dbus\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:41.146648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146264 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-dbus\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:41.146648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146291 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-sys\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.146648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146316 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-lib-modules\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.146648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146323 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-registration-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.146648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146404 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ff9b33f-ec62-4d90-80dc-745c2ffa9498-host\") pod \"node-ca-brsvk\" (UID: \"9ff9b33f-ec62-4d90-80dc-745c2ffa9498\") " pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:41.146648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146423 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-sys\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.146648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/707c4468-3606-41ee-8a87-dc5764875fa5-env-overrides\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.146648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146492 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ff9b33f-ec62-4d90-80dc-745c2ffa9498-host\") pod \"node-ca-brsvk\" (UID: \"9ff9b33f-ec62-4d90-80dc-745c2ffa9498\") " pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:41.146648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146510 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fb6985f2-1b61-47fd-a1b0-17429a573c7d-konnectivity-ca\") pod \"konnectivity-agent-2nd7r\" (UID: \"fb6985f2-1b61-47fd-a1b0-17429a573c7d\") " pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:48:41.146648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146491 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-registration-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146676 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-etc-selinux\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146707 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-sys-fs\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146737 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4ebb3c3f-eda3-4c28-987b-11627a176748-cni-binary-copy\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146744 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkcsk\" (UniqueName: \"kubernetes.io/projected/9ff9b33f-ec62-4d90-80dc-745c2ffa9498-kube-api-access-dkcsk\") pod \"node-ca-brsvk\" (UID: \"9ff9b33f-ec62-4d90-80dc-745c2ffa9498\") " pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146807 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-host\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146843 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmkdp\" (UniqueName: \"kubernetes.io/projected/09e10046-6938-48cf-9a41-9a726dbc472f-kube-api-access-xmkdp\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146893 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4423aa72-95c7-4610-9d94-76b3f1c2b76e-hosts-file\") pod \"node-resolver-wxmxp\" (UID: \"4423aa72-95c7-4610-9d94-76b3f1c2b76e\") " pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146919 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-etc-openvswitch\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146946 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.146971 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km997\" (UniqueName: \"kubernetes.io/projected/4423aa72-95c7-4610-9d94-76b3f1c2b76e-kube-api-access-km997\") pod \"node-resolver-wxmxp\" (UID: \"4423aa72-95c7-4610-9d94-76b3f1c2b76e\") " pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147002 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-node-log\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147018 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/707c4468-3606-41ee-8a87-dc5764875fa5-env-overrides\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147031 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147070 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-etc-selinux\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147070 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq6j8\" (UniqueName: \"kubernetes.io/projected/707c4468-3606-41ee-8a87-dc5764875fa5-kube-api-access-fq6j8\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147112 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-kubelet-config\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:41.147148 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147122 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-host\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147143 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fb6985f2-1b61-47fd-a1b0-17429a573c7d-agent-certs\") pod \"konnectivity-agent-2nd7r\" (UID: \"fb6985f2-1b61-47fd-a1b0-17429a573c7d\") " pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147239 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldkgr\" (UniqueName: \"kubernetes.io/projected/c5c374b5-74a1-41b6-826c-288b284cc627-kube-api-access-ldkgr\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147247 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fb6985f2-1b61-47fd-a1b0-17429a573c7d-konnectivity-ca\") pod \"konnectivity-agent-2nd7r\" (UID: \"fb6985f2-1b61-47fd-a1b0-17429a573c7d\") " pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147269 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbsxc\" (UniqueName: \"kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc\") pod \"network-check-target-zdbst\" (UID: \"7de4f318-5759-4f4f-870e-01b72f5f3e90\") " pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147307 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-etc-openvswitch\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147313 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-kubernetes\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147075 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147312 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4423aa72-95c7-4610-9d94-76b3f1c2b76e-hosts-file\") pod \"node-resolver-wxmxp\" (UID: \"4423aa72-95c7-4610-9d94-76b3f1c2b76e\") " pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147341 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-sysctl-d\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147276 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/707c4468-3606-41ee-8a87-dc5764875fa5-ovnkube-config\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147362 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-sys-fs\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147406 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147412 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-kubelet-config\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147421 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-node-log\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147436 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-sysctl-conf\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147174 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/707c4468-3606-41ee-8a87-dc5764875fa5-ovnkube-script-lib\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147438 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-sysctl-d\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147534 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-kubernetes\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147616 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-sysctl-conf\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147682 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147749 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-device-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147767 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ff9b33f-ec62-4d90-80dc-745c2ffa9498-serviceca\") pod \"node-ca-brsvk\" (UID: \"9ff9b33f-ec62-4d90-80dc-745c2ffa9498\") " pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147783 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4ebb3c3f-eda3-4c28-987b-11627a176748-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147801 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-run-netns\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147825 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-var-lib-openvswitch\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147840 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-device-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147883 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-run-ovn\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.147953 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-run\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148015 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-var-lib-openvswitch\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148015 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-run-netns\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148055 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-run\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148097 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-log-socket\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148122 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-cni-bin\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.148909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148155 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-run-ovn\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148174 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-socket-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148203 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-cnibin\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148230 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7g6b\" (UniqueName: \"kubernetes.io/projected/4ebb3c3f-eda3-4c28-987b-11627a176748-kube-api-access-w7g6b\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148208 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148256 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-kubelet\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148206 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-log-socket\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148305 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ff9b33f-ec62-4d90-80dc-745c2ffa9498-serviceca\") pod \"node-ca-brsvk\" (UID: \"9ff9b33f-ec62-4d90-80dc-745c2ffa9498\") " pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148319 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148317 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-cni-bin\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148345 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4ebb3c3f-eda3-4c28-987b-11627a176748-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148380 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-socket-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.148393 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.148464 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148413 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-kubelet\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148436 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c32c893-7163-4346-a810-ac3b1aed5eeb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148554 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-cnibin\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.149452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148381 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4ebb3c3f-eda3-4c28-987b-11627a176748-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.148607 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret podName:3dee8e4b-8ebb-470a-aaac-a610ba92c6dd nodeName:}" failed. No retries permitted until 2026-07-06 23:48:41.648567265 +0000 UTC m=+2.158728207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret") pod "global-pull-secret-syncer-8dl25" (UID: "3dee8e4b-8ebb-470a-aaac-a610ba92c6dd") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.148650 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs podName:c5c374b5-74a1-41b6-826c-288b284cc627 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:41.64863665 +0000 UTC m=+2.158797592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs") pod "network-metrics-daemon-c5zp8" (UID: "c5c374b5-74a1-41b6-826c-288b284cc627") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148679 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4423aa72-95c7-4610-9d94-76b3f1c2b76e-tmp-dir\") pod \"node-resolver-wxmxp\" (UID: \"4423aa72-95c7-4610-9d94-76b3f1c2b76e\") " pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148716 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-modprobe-d\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148722 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09e10046-6938-48cf-9a41-9a726dbc472f-tmp\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148744 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-var-lib-kubelet\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148768 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09e10046-6938-48cf-9a41-9a726dbc472f-etc-tuned\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148791 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-os-release\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148806 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-var-lib-kubelet\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148815 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-run-openvswitch\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148844 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-slash\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148864 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09e10046-6938-48cf-9a41-9a726dbc472f-etc-modprobe-d\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148890 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-run-ovn-kubernetes\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148915 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-run-systemd\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148924 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-run-openvswitch\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.149946 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148954 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4ebb3c3f-eda3-4c28-987b-11627a176748-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.150391 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.148989 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-run-systemd\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.150391 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.149009 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-slash\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.150391 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.149027 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/707c4468-3606-41ee-8a87-dc5764875fa5-host-run-ovn-kubernetes\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.150391 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.149016 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4423aa72-95c7-4610-9d94-76b3f1c2b76e-tmp-dir\") pod \"node-resolver-wxmxp\" (UID: \"4423aa72-95c7-4610-9d94-76b3f1c2b76e\") " pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:41.150391 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.149067 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4ebb3c3f-eda3-4c28-987b-11627a176748-os-release\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.150391 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.149717 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/707c4468-3606-41ee-8a87-dc5764875fa5-ovn-node-metrics-cert\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.150669 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.150652 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fb6985f2-1b61-47fd-a1b0-17429a573c7d-agent-certs\") pod \"konnectivity-agent-2nd7r\" (UID: \"fb6985f2-1b61-47fd-a1b0-17429a573c7d\") " pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:48:41.150931 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.150913 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09e10046-6938-48cf-9a41-9a726dbc472f-etc-tuned\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.160898 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.160876 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7g6b\" (UniqueName: \"kubernetes.io/projected/4ebb3c3f-eda3-4c28-987b-11627a176748-kube-api-access-w7g6b\") pod \"multus-additional-cni-plugins-tw7t9\" (UID: \"4ebb3c3f-eda3-4c28-987b-11627a176748\") " pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.160986 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.160973 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldkgr\" (UniqueName: \"kubernetes.io/projected/c5c374b5-74a1-41b6-826c-288b284cc627-kube-api-access-ldkgr\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:41.163760 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.163745 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:41.163760 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.163761 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:41.163892 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.163769 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rbsxc for pod openshift-network-diagnostics/network-check-target-zdbst: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:41.163892 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.163817 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc podName:7de4f318-5759-4f4f-870e-01b72f5f3e90 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:41.663805106 +0000 UTC m=+2.173966040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rbsxc" (UniqueName: "kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc") pod "network-check-target-zdbst" (UID: "7de4f318-5759-4f4f-870e-01b72f5f3e90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:41.164689 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.164668 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsr5s\" (UniqueName: \"kubernetes.io/projected/6c32c893-7163-4346-a810-ac3b1aed5eeb-kube-api-access-vsr5s\") pod \"aws-ebs-csi-driver-node-fkchr\" (UID: \"6c32c893-7163-4346-a810-ac3b1aed5eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.165268 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.165251 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmkdp\" (UniqueName: \"kubernetes.io/projected/09e10046-6938-48cf-9a41-9a726dbc472f-kube-api-access-xmkdp\") pod \"tuned-hs2xd\" (UID: \"09e10046-6938-48cf-9a41-9a726dbc472f\") " pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.166469 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.166452 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-km997\" (UniqueName: \"kubernetes.io/projected/4423aa72-95c7-4610-9d94-76b3f1c2b76e-kube-api-access-km997\") pod \"node-resolver-wxmxp\" (UID: \"4423aa72-95c7-4610-9d94-76b3f1c2b76e\") " pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:41.168405 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.168385 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkcsk\" (UniqueName: \"kubernetes.io/projected/9ff9b33f-ec62-4d90-80dc-745c2ffa9498-kube-api-access-dkcsk\") pod \"node-ca-brsvk\" (UID: \"9ff9b33f-ec62-4d90-80dc-745c2ffa9498\") " pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:41.169100 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.169084 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq6j8\" (UniqueName: \"kubernetes.io/projected/707c4468-3606-41ee-8a87-dc5764875fa5-kube-api-access-fq6j8\") pod \"ovnkube-node-2z7v2\" (UID: \"707c4468-3606-41ee-8a87-dc5764875fa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.259965 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.259885 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z6dqd" Jul 06 23:48:41.265433 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.265408 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-24nhh" Jul 06 23:48:41.272107 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:41.272081 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda58005e6_88fa_4cda_bf75_bff326b46b7c.slice/crio-8aa46b7531141303f2d8cbc45aff328c017ab737d19e48d734f4b2e921c4140b WatchSource:0}: Error finding container 8aa46b7531141303f2d8cbc45aff328c017ab737d19e48d734f4b2e921c4140b: Status 404 returned error can't find the container with id 8aa46b7531141303f2d8cbc45aff328c017ab737d19e48d734f4b2e921c4140b Jul 06 23:48:41.274066 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.274047 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:41.275037 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:41.275016 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847d0bbc_bde0_4006_ad49_b87007f8923c.slice/crio-ede8a3642168245f85a7351a771ed6c48c327195fe8ac1d1d7349b2288ecfa2c WatchSource:0}: Error finding container ede8a3642168245f85a7351a771ed6c48c327195fe8ac1d1d7349b2288ecfa2c: Status 404 returned error can't find the container with id ede8a3642168245f85a7351a771ed6c48c327195fe8ac1d1d7349b2288ecfa2c Jul 06 23:48:41.288601 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.288575 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:48:41.293819 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.293802 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" Jul 06 23:48:41.298734 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:41.298709 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6985f2_1b61_47fd_a1b0_17429a573c7d.slice/crio-a03a1f1f7c8a2efba85fa45a0956e1ca99f1691d45157433b4b5cdd7a7bf2b29 WatchSource:0}: Error finding container a03a1f1f7c8a2efba85fa45a0956e1ca99f1691d45157433b4b5cdd7a7bf2b29: Status 404 returned error can't find the container with id a03a1f1f7c8a2efba85fa45a0956e1ca99f1691d45157433b4b5cdd7a7bf2b29 Jul 06 23:48:41.303099 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:41.303068 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c32c893_7163_4346_a810_ac3b1aed5eeb.slice/crio-c1823313587c02dbab7fa5ab0c9745bf9147b2587070d1d85803825ea0fef470 WatchSource:0}: Error finding container c1823313587c02dbab7fa5ab0c9745bf9147b2587070d1d85803825ea0fef470: Status 404 returned error can't find the container with id c1823313587c02dbab7fa5ab0c9745bf9147b2587070d1d85803825ea0fef470 Jul 06 23:48:41.309539 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.309517 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-brsvk" Jul 06 23:48:41.320306 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:41.320276 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff9b33f_ec62_4d90_80dc_745c2ffa9498.slice/crio-744134e6ff8b3816927e44e8cf20ec4470c3e607574456a4c7d058ec2d46d679 WatchSource:0}: Error finding container 744134e6ff8b3816927e44e8cf20ec4470c3e607574456a4c7d058ec2d46d679: Status 404 returned error can't find the container with id 744134e6ff8b3816927e44e8cf20ec4470c3e607574456a4c7d058ec2d46d679 Jul 06 23:48:41.327395 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.327369 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" Jul 06 23:48:41.336770 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:41.336745 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ebb3c3f_eda3_4c28_987b_11627a176748.slice/crio-aba7b9b1e5a037071fef2c49bd5b3d564c9dbda569729f2adeeae8de37ee459e WatchSource:0}: Error finding container aba7b9b1e5a037071fef2c49bd5b3d564c9dbda569729f2adeeae8de37ee459e: Status 404 returned error can't find the container with id aba7b9b1e5a037071fef2c49bd5b3d564c9dbda569729f2adeeae8de37ee459e Jul 06 23:48:41.355404 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.355380 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" Jul 06 23:48:41.360672 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.360647 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wxmxp" Jul 06 23:48:41.366265 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.366090 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:48:41.373810 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:41.373784 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4423aa72_95c7_4610_9d94_76b3f1c2b76e.slice/crio-9ae8da61a09dfbd9881925ff4118ee84c6c9d9327f8fe009e900c33195526f88 WatchSource:0}: Error finding container 9ae8da61a09dfbd9881925ff4118ee84c6c9d9327f8fe009e900c33195526f88: Status 404 returned error can't find the container with id 9ae8da61a09dfbd9881925ff4118ee84c6c9d9327f8fe009e900c33195526f88 Jul 06 23:48:41.376145 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:48:41.375973 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707c4468_3606_41ee_8a87_dc5764875fa5.slice/crio-7ecc7348a825fb9935f17d687d6ab972c165d841deb95c79078333a5549ef3a5 WatchSource:0}: Error finding container 7ecc7348a825fb9935f17d687d6ab972c165d841deb95c79078333a5549ef3a5: Status 404 returned error can't find the container with id 7ecc7348a825fb9935f17d687d6ab972c165d841deb95c79078333a5549ef3a5 Jul 06 23:48:41.502633 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.502593 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:41.652928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.652829 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:41.652928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.652914 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:41.653175 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.653111 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:41.653175 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.653152 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:41.653283 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.653189 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs podName:c5c374b5-74a1-41b6-826c-288b284cc627 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:42.653164871 +0000 UTC m=+3.163325812 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs") pod "network-metrics-daemon-c5zp8" (UID: "c5c374b5-74a1-41b6-826c-288b284cc627") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:41.653283 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.653218 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret podName:3dee8e4b-8ebb-470a-aaac-a610ba92c6dd nodeName:}" failed. No retries permitted until 2026-07-06 23:48:42.653199839 +0000 UTC m=+3.163360781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret") pod "global-pull-secret-syncer-8dl25" (UID: "3dee8e4b-8ebb-470a-aaac-a610ba92c6dd") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:41.754368 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.754325 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbsxc\" (UniqueName: \"kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc\") pod \"network-check-target-zdbst\" (UID: \"7de4f318-5759-4f4f-870e-01b72f5f3e90\") " pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:41.754571 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.754527 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:41.754571 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.754550 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:41.754571 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.754564 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rbsxc for pod openshift-network-diagnostics/network-check-target-zdbst: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:41.754749 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:41.754630 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc podName:7de4f318-5759-4f4f-870e-01b72f5f3e90 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:42.7546048 +0000 UTC m=+3.264765739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbsxc" (UniqueName: "kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc") pod "network-check-target-zdbst" (UID: "7de4f318-5759-4f4f-870e-01b72f5f3e90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:41.995802 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.995693 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:43:41 +0000 UTC" deadline="2027-12-27 05:56:17.945021658 +0000 UTC" Jul 06 23:48:41.995802 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:41.995737 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12918h7m35.949290005s" Jul 06 23:48:42.079067 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.078942 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal" event={"ID":"373575230b88bb4ec2f11847d42c51bd","Type":"ContainerStarted","Data":"85242e81115ae7ac54473434ab7399ee77b3b799cb48cba4d0bc1791b189a039"} Jul 06 23:48:42.087886 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.087803 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wxmxp" event={"ID":"4423aa72-95c7-4610-9d94-76b3f1c2b76e","Type":"ContainerStarted","Data":"9ae8da61a09dfbd9881925ff4118ee84c6c9d9327f8fe009e900c33195526f88"} Jul 06 23:48:42.103725 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.103607 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-brsvk" event={"ID":"9ff9b33f-ec62-4d90-80dc-745c2ffa9498","Type":"ContainerStarted","Data":"744134e6ff8b3816927e44e8cf20ec4470c3e607574456a4c7d058ec2d46d679"} Jul 06 23:48:42.125445 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.125400 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" event={"ID":"a9354effc7df6757b13f443e5e88f79e","Type":"ContainerStarted","Data":"bfad68cae46d484489bbf3a8dc19b5eb2c8ee203e089cc85b458d96e4ddd9c41"} Jul 06 23:48:42.142873 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.142653 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerStarted","Data":"7ecc7348a825fb9935f17d687d6ab972c165d841deb95c79078333a5549ef3a5"} Jul 06 23:48:42.151401 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.151354 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" event={"ID":"09e10046-6938-48cf-9a41-9a726dbc472f","Type":"ContainerStarted","Data":"d51286ec5618f391486a8fbdefb04cac815d6b214d56571ce968553d6dcd45aa"} Jul 06 23:48:42.179240 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.179196 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" event={"ID":"4ebb3c3f-eda3-4c28-987b-11627a176748","Type":"ContainerStarted","Data":"aba7b9b1e5a037071fef2c49bd5b3d564c9dbda569729f2adeeae8de37ee459e"} Jul 06 23:48:42.191752 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.191710 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" event={"ID":"6c32c893-7163-4346-a810-ac3b1aed5eeb","Type":"ContainerStarted","Data":"c1823313587c02dbab7fa5ab0c9745bf9147b2587070d1d85803825ea0fef470"} Jul 06 23:48:42.210489 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.210443 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2nd7r" event={"ID":"fb6985f2-1b61-47fd-a1b0-17429a573c7d","Type":"ContainerStarted","Data":"a03a1f1f7c8a2efba85fa45a0956e1ca99f1691d45157433b4b5cdd7a7bf2b29"} Jul 06 23:48:42.218877 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.218829 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-24nhh" event={"ID":"847d0bbc-bde0-4006-ad49-b87007f8923c","Type":"ContainerStarted","Data":"ede8a3642168245f85a7351a771ed6c48c327195fe8ac1d1d7349b2288ecfa2c"} Jul 06 23:48:42.225651 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.225609 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z6dqd" event={"ID":"a58005e6-88fa-4cda-bf75-bff326b46b7c","Type":"ContainerStarted","Data":"8aa46b7531141303f2d8cbc45aff328c017ab737d19e48d734f4b2e921c4140b"} Jul 06 23:48:42.259872 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.259764 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:42.670040 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.669043 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:42.670040 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.669108 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:42.670040 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:42.669273 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:42.670040 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:42.669341 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret podName:3dee8e4b-8ebb-470a-aaac-a610ba92c6dd nodeName:}" failed. No retries permitted until 2026-07-06 23:48:44.66932267 +0000 UTC m=+5.179483609 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret") pod "global-pull-secret-syncer-8dl25" (UID: "3dee8e4b-8ebb-470a-aaac-a610ba92c6dd") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:42.670040 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:42.669833 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:42.670040 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:42.669917 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs podName:c5c374b5-74a1-41b6-826c-288b284cc627 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:44.66989607 +0000 UTC m=+5.180057010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs") pod "network-metrics-daemon-c5zp8" (UID: "c5c374b5-74a1-41b6-826c-288b284cc627") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:42.769579 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.769536 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbsxc\" (UniqueName: \"kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc\") pod \"network-check-target-zdbst\" (UID: \"7de4f318-5759-4f4f-870e-01b72f5f3e90\") " pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:42.769792 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:42.769754 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:42.769792 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:42.769779 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:42.769792 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:42.769792 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rbsxc for pod openshift-network-diagnostics/network-check-target-zdbst: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:42.769957 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:42.769857 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc podName:7de4f318-5759-4f4f-870e-01b72f5f3e90 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:44.769835175 +0000 UTC m=+5.279996115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbsxc" (UniqueName: "kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc") pod "network-check-target-zdbst" (UID: "7de4f318-5759-4f4f-870e-01b72f5f3e90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:42.996477 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.996282 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:43:41 +0000 UTC" deadline="2027-11-30 19:41:38.231222568 +0000 UTC" Jul 06 23:48:42.996477 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:42.996353 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12283h52m55.234874661s" Jul 06 23:48:43.046379 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:43.045451 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:43.046379 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:43.045590 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:48:43.046379 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:43.046042 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:43.046379 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:43.046140 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:48:43.046379 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:43.046205 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:43.046379 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:43.046287 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:48:43.965337 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:43.965049 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:44.687508 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:44.687471 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:44.687995 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:44.687528 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:44.687995 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:44.687698 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:44.687995 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:44.687763 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret podName:3dee8e4b-8ebb-470a-aaac-a610ba92c6dd nodeName:}" failed. No retries permitted until 2026-07-06 23:48:48.687743815 +0000 UTC m=+9.197904756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret") pod "global-pull-secret-syncer-8dl25" (UID: "3dee8e4b-8ebb-470a-aaac-a610ba92c6dd") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:44.688210 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:44.688186 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:44.688287 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:44.688243 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs podName:c5c374b5-74a1-41b6-826c-288b284cc627 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:48.688226559 +0000 UTC m=+9.198387514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs") pod "network-metrics-daemon-c5zp8" (UID: "c5c374b5-74a1-41b6-826c-288b284cc627") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:44.788731 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:44.788090 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbsxc\" (UniqueName: \"kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc\") pod \"network-check-target-zdbst\" (UID: \"7de4f318-5759-4f4f-870e-01b72f5f3e90\") " pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:44.788731 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:44.788292 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:44.788731 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:44.788318 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:44.788731 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:44.788331 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rbsxc for pod openshift-network-diagnostics/network-check-target-zdbst: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:44.788731 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:44.788387 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc podName:7de4f318-5759-4f4f-870e-01b72f5f3e90 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:48.78836831 +0000 UTC m=+9.298529262 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbsxc" (UniqueName: "kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc") pod "network-check-target-zdbst" (UID: "7de4f318-5759-4f4f-870e-01b72f5f3e90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:45.046986 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:45.046145 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:45.046986 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:45.046272 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:48:45.046986 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:45.046649 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:45.046986 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:45.046740 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:48:45.046986 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:45.046787 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:45.046986 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:45.046888 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:48:47.045967 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:47.045909 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:47.045967 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:47.045970 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:47.046482 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:47.045919 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:47.046482 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:47.046080 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:48:47.046482 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:47.046179 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:48:47.046482 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:47.046302 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:48:48.720447 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:48.719743 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:48.720447 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:48.719809 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:48.720447 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:48.719940 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:48.720447 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:48.719992 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret podName:3dee8e4b-8ebb-470a-aaac-a610ba92c6dd nodeName:}" failed. No retries permitted until 2026-07-06 23:48:56.719971781 +0000 UTC m=+17.230132720 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret") pod "global-pull-secret-syncer-8dl25" (UID: "3dee8e4b-8ebb-470a-aaac-a610ba92c6dd") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:48.720447 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:48.720366 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:48.720447 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:48.720416 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs podName:c5c374b5-74a1-41b6-826c-288b284cc627 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:56.720401398 +0000 UTC m=+17.230562336 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs") pod "network-metrics-daemon-c5zp8" (UID: "c5c374b5-74a1-41b6-826c-288b284cc627") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:48.820472 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:48.820384 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbsxc\" (UniqueName: \"kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc\") pod \"network-check-target-zdbst\" (UID: \"7de4f318-5759-4f4f-870e-01b72f5f3e90\") " pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:48.820644 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:48.820579 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:48.820644 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:48.820606 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:48.820644 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:48.820620 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rbsxc for pod openshift-network-diagnostics/network-check-target-zdbst: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:48.820800 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:48.820676 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc podName:7de4f318-5759-4f4f-870e-01b72f5f3e90 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:56.820658074 +0000 UTC m=+17.330819012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbsxc" (UniqueName: "kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc") pod "network-check-target-zdbst" (UID: "7de4f318-5759-4f4f-870e-01b72f5f3e90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:49.045606 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:49.045571 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:49.045760 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:49.045706 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:48:49.045803 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:49.045773 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:49.045944 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:49.045912 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:48:49.046035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:49.046025 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:49.046093 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:49.046081 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:48:51.046062 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:51.046029 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:51.046495 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:51.046149 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:48:51.046495 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:51.046468 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:51.046594 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:51.046547 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:48:51.046594 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:51.046583 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:51.046673 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:51.046635 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:48:53.045864 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:53.045804 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:53.046330 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:53.045879 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:53.046330 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:53.045948 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:48:53.046330 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:53.045879 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:53.046330 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:53.045987 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:48:53.046330 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:53.046052 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:48:55.045225 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:55.045187 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:55.045736 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:55.045317 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:55.045736 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:55.045322 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:55.045736 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:55.045323 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:48:55.045736 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:55.045436 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:48:55.045736 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:55.045535 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:48:56.781777 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:56.781739 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:56.782283 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:56.781802 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:56.782283 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:56.781927 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:56.782283 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:56.781942 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:56.782283 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:56.782004 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret podName:3dee8e4b-8ebb-470a-aaac-a610ba92c6dd nodeName:}" failed. No retries permitted until 2026-07-06 23:49:12.781986686 +0000 UTC m=+33.292147621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret") pod "global-pull-secret-syncer-8dl25" (UID: "3dee8e4b-8ebb-470a-aaac-a610ba92c6dd") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:56.782283 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:56.782025 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs podName:c5c374b5-74a1-41b6-826c-288b284cc627 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:12.78201509 +0000 UTC m=+33.292176032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs") pod "network-metrics-daemon-c5zp8" (UID: "c5c374b5-74a1-41b6-826c-288b284cc627") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:56.882445 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:56.882399 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbsxc\" (UniqueName: \"kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc\") pod \"network-check-target-zdbst\" (UID: \"7de4f318-5759-4f4f-870e-01b72f5f3e90\") " pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:56.882620 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:56.882558 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:56.882620 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:56.882586 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:56.882620 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:56.882600 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rbsxc for pod openshift-network-diagnostics/network-check-target-zdbst: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:56.882769 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:56.882669 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc podName:7de4f318-5759-4f4f-870e-01b72f5f3e90 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:12.882650634 +0000 UTC m=+33.392811573 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbsxc" (UniqueName: "kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc") pod "network-check-target-zdbst" (UID: "7de4f318-5759-4f4f-870e-01b72f5f3e90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:57.046102 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:57.046018 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:57.046251 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:57.046018 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:57.046251 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:57.046153 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:48:57.046363 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:57.046249 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:48:57.046363 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:57.046018 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:57.046451 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:57.046392 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:48:59.045795 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:59.045749 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:48:59.045795 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:59.045778 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:48:59.046337 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:48:59.045774 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:48:59.046337 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:59.045900 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:48:59.046337 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:59.046062 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:48:59.046337 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:48:59.046157 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:49:01.045569 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:01.045531 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:01.046089 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:01.045575 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:01.046089 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:01.045538 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:01.046089 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:01.045670 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:49:01.046089 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:01.045743 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:49:01.046089 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:01.045825 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:49:02.268117 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:02.268081 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerStarted","Data":"5a6fe544ccc23c182738952013f7096600abaa316d01307371c6c4db729baf13"} Jul 06 23:49:02.268509 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:02.268160 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerStarted","Data":"214b991963ad96b40a430b7a6e98145c869ce653a4eb07ae51c898a4462e71a0"} Jul 06 23:49:02.269823 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:02.269789 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" event={"ID":"09e10046-6938-48cf-9a41-9a726dbc472f","Type":"ContainerStarted","Data":"3f6f726d4680272428e98a97e38053d44b7a859e6926505058b7d6264f377947"} Jul 06 23:49:02.271175 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:02.271150 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-24nhh" event={"ID":"847d0bbc-bde0-4006-ad49-b87007f8923c","Type":"ContainerStarted","Data":"64f67f940f5b743cc8cfc105e51bbf996444e1e052be5e3a55f2945e77a14aa1"} Jul 06 23:49:02.272451 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:02.272430 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal" event={"ID":"373575230b88bb4ec2f11847d42c51bd","Type":"ContainerStarted","Data":"4ebb0f9da910c4be48ff003ba725ca4dc2bfbab4cca4ab6634e32dd04654da3d"} Jul 06 23:49:02.286234 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:02.286190 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hs2xd" podStartSLOduration=1.848179266 podStartE2EDuration="22.286176549s" totalImagesPullingTime="20.437997283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:02.285869591 +0000 UTC m=+22.796030540" watchObservedRunningTime="2026-07-06 23:49:02.286176549 +0000 UTC m=+22.796337507" Jul 06 23:49:02.324925 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:02.324885 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-213.ec2.internal" podStartSLOduration=21.32487032 podStartE2EDuration="21.32487032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:02.300653205 +0000 UTC m=+22.810814161" watchObservedRunningTime="2026-07-06 23:49:02.32487032 +0000 UTC m=+22.835031271" Jul 06 23:49:02.325119 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:02.325098 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-24nhh" podStartSLOduration=1.7108488899999998 podStartE2EDuration="22.32509206s" totalImagesPullingTime="20.61424317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:02.323596251 +0000 UTC m=+22.833757226" watchObservedRunningTime="2026-07-06 23:49:02.32509206 +0000 UTC m=+22.835253017" Jul 06 23:49:03.046354 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.046159 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:03.046527 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.046173 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:03.046527 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:03.046441 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:49:03.046619 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:03.046528 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:49:03.046619 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.046193 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:03.046701 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:03.046646 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:49:03.277979 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.277954 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 06 23:49:03.278651 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.278309 2558 generic.go:356] "Generic (PLEG): container finished" podID="707c4468-3606-41ee-8a87-dc5764875fa5" containerID="5a6fe544ccc23c182738952013f7096600abaa316d01307371c6c4db729baf13" exitCode=1 Jul 06 23:49:03.278651 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.278379 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerDied","Data":"5a6fe544ccc23c182738952013f7096600abaa316d01307371c6c4db729baf13"} Jul 06 23:49:03.278651 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.278455 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerStarted","Data":"39e009a673609188be7655f6bac32567e0e97cf280869c905becab89e648e5e4"} Jul 06 23:49:03.278651 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.278486 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerStarted","Data":"6a7c68a7a1a86ec8beaaeeefaad3583cce31e3b9073a1e13f2cda88a62cf8a5e"} Jul 06 23:49:03.278651 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.278520 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerStarted","Data":"2d874c22d76bc6bab35f19ffa05620109a7a22f474a99de4292d97fa9f81d304"} Jul 06 23:49:03.278651 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.278547 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerStarted","Data":"6b33cecb52fe5ea30eabc30f35040741a780fe7be3cfaa0606486fccab7c2231"} Jul 06 23:49:03.279885 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.279835 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z6dqd" event={"ID":"a58005e6-88fa-4cda-bf75-bff326b46b7c","Type":"ContainerStarted","Data":"3d1b5a5f244fb3ddfdfc165e8910ed3ded8c5927d3328ef33cb62e9e1cdeda27"} Jul 06 23:49:03.295375 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:03.295325 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z6dqd" podStartSLOduration=2.762467934 podStartE2EDuration="23.295309316s" totalImagesPullingTime="20.532841382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:03.294831 +0000 UTC m=+23.804991957" watchObservedRunningTime="2026-07-06 23:49:03.295309316 +0000 UTC m=+23.805470274" Jul 06 23:49:04.282901 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.282867 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" event={"ID":"6c32c893-7163-4346-a810-ac3b1aed5eeb","Type":"ContainerStarted","Data":"7d3f42669f66f1d02fc4e2912d9f7fa6a9f8dabdfcdce3739170b6b7d719cf74"} Jul 06 23:49:04.284159 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.284133 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2nd7r" event={"ID":"fb6985f2-1b61-47fd-a1b0-17429a573c7d","Type":"ContainerStarted","Data":"d8584d184c0f598913721a7d43d3719342d24c28964a1bb6ca3a3da3a16277b5"} Jul 06 23:49:04.285483 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.285458 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wxmxp" event={"ID":"4423aa72-95c7-4610-9d94-76b3f1c2b76e","Type":"ContainerStarted","Data":"dc0e217833873081852016d6632c21a72e5380ac93c131e01be8e62532124548"} Jul 06 23:49:04.286688 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.286666 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-brsvk" event={"ID":"9ff9b33f-ec62-4d90-80dc-745c2ffa9498","Type":"ContainerStarted","Data":"7b08135f098754e362b9ed274cbc00ef51307206008eb835f3b9c86d73299c0b"} Jul 06 23:49:04.288046 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.288019 2558 generic.go:356] "Generic (PLEG): container finished" podID="a9354effc7df6757b13f443e5e88f79e" containerID="e3b2000beb2f7851c03e360ce38704d0341c7e4886c2fd08f2115432b44578d2" exitCode=0 Jul 06 23:49:04.288144 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.288101 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" event={"ID":"a9354effc7df6757b13f443e5e88f79e","Type":"ContainerDied","Data":"e3b2000beb2f7851c03e360ce38704d0341c7e4886c2fd08f2115432b44578d2"} Jul 06 23:49:04.289599 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.289580 2558 generic.go:356] "Generic (PLEG): container finished" podID="4ebb3c3f-eda3-4c28-987b-11627a176748" containerID="78715d37faef2e73880e53ba7a06217a075355c08370b47ab89c641d3a0c6d17" exitCode=0 Jul 06 23:49:04.289693 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.289656 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" event={"ID":"4ebb3c3f-eda3-4c28-987b-11627a176748","Type":"ContainerDied","Data":"78715d37faef2e73880e53ba7a06217a075355c08370b47ab89c641d3a0c6d17"} Jul 06 23:49:04.331977 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.331931 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2nd7r" podStartSLOduration=3.773798865 podStartE2EDuration="24.33189977s" totalImagesPullingTime="20.558100905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:04.30626281 +0000 UTC m=+24.816423786" watchObservedRunningTime="2026-07-06 23:49:04.33189977 +0000 UTC m=+24.842060727" Jul 06 23:49:04.332125 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.332072 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-brsvk" podStartSLOduration=3.8477750779999997 podStartE2EDuration="24.332065871s" totalImagesPullingTime="20.484290793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:04.331546793 +0000 UTC m=+24.841707750" watchObservedRunningTime="2026-07-06 23:49:04.332065871 +0000 UTC m=+24.842226827" Jul 06 23:49:04.360538 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.360483 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wxmxp" podStartSLOduration=3.876550555 podStartE2EDuration="24.360448336s" totalImagesPullingTime="20.483897781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:04.359581183 +0000 UTC m=+24.869742140" watchObservedRunningTime="2026-07-06 23:49:04.360448336 +0000 UTC m=+24.870609294" Jul 06 23:49:04.728888 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:04.728830 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 23:49:05.008311 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.007998 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:49:04.728878167Z","UUID":"5cbb9857-8e80-41d6-911d-ad01420f3ffa","Handler":null,"Name":"","Endpoint":""} Jul 06 23:49:05.009837 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.009815 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 23:49:05.009975 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.009889 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 23:49:05.045261 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.045229 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:05.045261 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.045250 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:05.045539 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.045344 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:05.045539 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:05.045363 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:49:05.045539 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:05.045460 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:49:05.045686 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:05.045541 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:49:05.293368 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.293332 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" event={"ID":"6c32c893-7163-4346-a810-ac3b1aed5eeb","Type":"ContainerStarted","Data":"4fc9fc06b1643ca36f685833b9d4afc9e61fef6a2a2052f0a553016d59fd7761"} Jul 06 23:49:05.295729 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.295693 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" event={"ID":"a9354effc7df6757b13f443e5e88f79e","Type":"ContainerStarted","Data":"769223c73e7279d13e4eeaa9608e66574334bd6e3a062ae01be18297a7fc717e"} Jul 06 23:49:05.299339 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.299320 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 06 23:49:05.299793 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.299770 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerStarted","Data":"814ac4a744cf0a836b95f1632e4e794f165bf23fff24ff2294473bccef54f789"} Jul 06 23:49:05.316409 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:05.316348 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-213.ec2.internal" podStartSLOduration=24.316326056 podStartE2EDuration="24.316326056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:05.316318183 +0000 UTC m=+25.826479141" watchObservedRunningTime="2026-07-06 23:49:05.316326056 +0000 UTC m=+25.826487013" Jul 06 23:49:06.312129 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:06.312101 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:49:06.312960 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:06.312942 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:49:07.045789 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:07.045750 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:07.045989 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:07.045765 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:07.045989 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:07.045905 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:07.045989 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:07.045902 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:49:07.046128 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:07.045994 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:49:07.046128 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:07.046081 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:49:07.115839 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:07.115803 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:49:07.116371 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:07.116343 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2nd7r" Jul 06 23:49:07.307836 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:07.307748 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" event={"ID":"6c32c893-7163-4346-a810-ac3b1aed5eeb","Type":"ContainerStarted","Data":"f72e1fd60d92b5751acd1b50380a41326cae876e8ccd012cd97898beb02d263f"} Jul 06 23:49:07.332018 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:07.331967 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkchr" podStartSLOduration=4.233690408 podStartE2EDuration="27.331952475s" totalImagesPullingTime="23.098262067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:07.331415292 +0000 UTC m=+27.841576251" watchObservedRunningTime="2026-07-06 23:49:07.331952475 +0000 UTC m=+27.842113432" Jul 06 23:49:09.046325 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.046147 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:09.046325 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.046147 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:09.046875 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.046147 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:09.046875 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:09.046462 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:49:09.046875 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:09.046469 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:49:09.046875 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:09.046517 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:49:09.315639 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.315562 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 06 23:49:09.315981 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.315955 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerStarted","Data":"186ebd84fd1bb770f6644967a47fcc48d3ac91c76a3003a716735280c1397ea0"} Jul 06 23:49:09.316358 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.316224 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:49:09.316358 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.316298 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:49:09.316508 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.316388 2558 scope.go:117] "RemoveContainer" containerID="5a6fe544ccc23c182738952013f7096600abaa316d01307371c6c4db729baf13" Jul 06 23:49:09.317883 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.317832 2558 generic.go:356] "Generic (PLEG): container finished" podID="4ebb3c3f-eda3-4c28-987b-11627a176748" containerID="cffa179a25cd69951df3360fadcae7c27e8eedebdbce127f0441e46b323e8a86" exitCode=0 Jul 06 23:49:09.317979 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.317881 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" event={"ID":"4ebb3c3f-eda3-4c28-987b-11627a176748","Type":"ContainerDied","Data":"cffa179a25cd69951df3360fadcae7c27e8eedebdbce127f0441e46b323e8a86"} Jul 06 23:49:09.333651 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:09.333627 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:49:10.323576 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.323401 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 06 23:49:10.324021 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.323930 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" event={"ID":"707c4468-3606-41ee-8a87-dc5764875fa5","Type":"ContainerStarted","Data":"5febcc6cebb1f3dbe6ab686f8aa79486d513d5100d1209339aee85c94603737c"} Jul 06 23:49:10.324268 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.324243 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:49:10.326147 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.326126 2558 generic.go:356] "Generic (PLEG): container finished" podID="4ebb3c3f-eda3-4c28-987b-11627a176748" containerID="6489542418c2c02a4bdda0e8c78ffbea1c53c6f3c618045c4b56bf9f2a1b4d54" exitCode=0 Jul 06 23:49:10.326278 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.326171 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" event={"ID":"4ebb3c3f-eda3-4c28-987b-11627a176748","Type":"ContainerDied","Data":"6489542418c2c02a4bdda0e8c78ffbea1c53c6f3c618045c4b56bf9f2a1b4d54"} Jul 06 23:49:10.341112 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.341085 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:49:10.353382 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.353327 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" podStartSLOduration=9.691177764 podStartE2EDuration="30.35330987s" totalImagesPullingTime="20.662132106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:10.35263024 +0000 UTC m=+30.862791197" watchObservedRunningTime="2026-07-06 23:49:10.35330987 +0000 UTC m=+30.863470826" Jul 06 23:49:10.490279 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.490247 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zdbst"] Jul 06 23:49:10.490447 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.490394 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:10.490493 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:10.490473 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:49:10.493018 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.492984 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8dl25"] Jul 06 23:49:10.493165 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.493112 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:10.493225 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:10.493205 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:49:10.493784 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.493755 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c5zp8"] Jul 06 23:49:10.493926 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:10.493879 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:10.493977 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:10.493954 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:49:12.045616 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:12.045575 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:12.046208 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:12.045575 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:12.046208 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.045710 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:49:12.046208 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:12.045575 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:12.046208 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.045766 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:49:12.046208 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.045889 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:49:12.332544 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:12.332449 2558 generic.go:356] "Generic (PLEG): container finished" podID="4ebb3c3f-eda3-4c28-987b-11627a176748" containerID="fd3916d52ac26307e971c6d0b4ab8899a5de95816ff833c4201c7af797e658ac" exitCode=0 Jul 06 23:49:12.332544 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:12.332523 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" event={"ID":"4ebb3c3f-eda3-4c28-987b-11627a176748","Type":"ContainerDied","Data":"fd3916d52ac26307e971c6d0b4ab8899a5de95816ff833c4201c7af797e658ac"} Jul 06 23:49:12.797283 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:12.797234 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:12.797482 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:12.797306 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:12.797482 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.797393 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:49:12.797482 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.797419 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:49:12.797482 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.797473 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs podName:c5c374b5-74a1-41b6-826c-288b284cc627 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:44.797457406 +0000 UTC m=+65.307618341 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs") pod "network-metrics-daemon-c5zp8" (UID: "c5c374b5-74a1-41b6-826c-288b284cc627") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:49:12.797698 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.797490 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret podName:3dee8e4b-8ebb-470a-aaac-a610ba92c6dd nodeName:}" failed. No retries permitted until 2026-07-06 23:49:44.79748299 +0000 UTC m=+65.307643924 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret") pod "global-pull-secret-syncer-8dl25" (UID: "3dee8e4b-8ebb-470a-aaac-a610ba92c6dd") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:49:12.898000 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:12.897965 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbsxc\" (UniqueName: \"kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc\") pod \"network-check-target-zdbst\" (UID: \"7de4f318-5759-4f4f-870e-01b72f5f3e90\") " pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:12.898173 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.898119 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:49:12.898173 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.898136 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:49:12.898173 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.898145 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rbsxc for pod openshift-network-diagnostics/network-check-target-zdbst: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:49:12.898294 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:12.898204 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc podName:7de4f318-5759-4f4f-870e-01b72f5f3e90 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:44.898186055 +0000 UTC m=+65.408347005 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbsxc" (UniqueName: "kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc") pod "network-check-target-zdbst" (UID: "7de4f318-5759-4f4f-870e-01b72f5f3e90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:49:14.045982 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:14.045949 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:14.046440 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:14.045948 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:14.046440 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:14.046079 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8dl25" podUID="3dee8e4b-8ebb-470a-aaac-a610ba92c6dd" Jul 06 23:49:14.046440 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:14.045949 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:14.046440 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:14.046203 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c5zp8" podUID="c5c374b5-74a1-41b6-826c-288b284cc627" Jul 06 23:49:14.046440 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:14.046233 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zdbst" podUID="7de4f318-5759-4f4f-870e-01b72f5f3e90" Jul 06 23:49:15.250441 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.250407 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-213.ec2.internal" event="NodeReady" Jul 06 23:49:15.250945 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.250556 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 23:49:15.285146 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.285112 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks"] Jul 06 23:49:15.290779 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.290747 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8"] Jul 06 23:49:15.290947 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.290922 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.294501 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.294366 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 23:49:15.294501 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.294445 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 23:49:15.294708 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.294640 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 23:49:15.294760 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.294709 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 23:49:15.295446 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.295423 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh"] Jul 06 23:49:15.295583 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.295543 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.298264 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.298232 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 23:49:15.298796 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.298778 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 23:49:15.298967 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.298953 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 23:49:15.299178 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.299160 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 23:49:15.299612 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.299589 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-649bbb5647-76qxk"] Jul 06 23:49:15.300126 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.300106 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" Jul 06 23:49:15.302725 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.302703 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 23:49:15.302823 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.302722 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-grrmf\"" Jul 06 23:49:15.303159 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.303141 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q2rzk"] Jul 06 23:49:15.303281 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.303264 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.305871 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.305829 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 23:49:15.306233 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.306215 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 23:49:15.306341 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.306320 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5vhf7\"" Jul 06 23:49:15.306442 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.306230 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 23:49:15.307892 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.307876 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks"] Jul 06 23:49:15.308067 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.308049 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-s8nvv"] Jul 06 23:49:15.308169 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.307998 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q2rzk" Jul 06 23:49:15.311481 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.311461 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 23:49:15.311773 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.311755 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 23:49:15.311965 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.311933 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 23:49:15.312771 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.312647 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hk7zr\"" Jul 06 23:49:15.312771 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.312707 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 23:49:15.313541 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.313489 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8"] Jul 06 23:49:15.313794 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.313780 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-649bbb5647-76qxk"] Jul 06 23:49:15.313948 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.313736 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.315194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.315176 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh"] Jul 06 23:49:15.316775 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.316754 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s8nvv"] Jul 06 23:49:15.335446 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.335409 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 23:49:15.336607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.336057 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p99x7\"" Jul 06 23:49:15.336607 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.336165 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 23:49:15.337924 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.337890 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q2rzk"] Jul 06 23:49:15.416744 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.416708 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-image-registry-private-configuration\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.416744 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.416748 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0031876a-4b74-4311-917d-8dba25b6df06-ca-trust-extracted\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.417009 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.416772 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn8wl\" (UniqueName: \"kubernetes.io/projected/87190854-ff54-4120-bb7d-58bed7aa53ea-kube-api-access-zn8wl\") pod \"klusterlet-addon-workmgr-d6694ffc5-l4vks\" (UID: \"87190854-ff54-4120-bb7d-58bed7aa53ea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.417009 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.416795 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87190854-ff54-4120-bb7d-58bed7aa53ea-tmp\") pod \"klusterlet-addon-workmgr-d6694ffc5-l4vks\" (UID: \"87190854-ff54-4120-bb7d-58bed7aa53ea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.417009 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.416815 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdf73de7-5618-48db-9e38-37e2b82f372b-cert\") pod \"ingress-canary-q2rzk\" (UID: \"cdf73de7-5618-48db-9e38-37e2b82f372b\") " pod="openshift-ingress-canary/ingress-canary-q2rzk" Jul 06 23:49:15.417009 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.416835 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbwzr\" (UniqueName: \"kubernetes.io/projected/cdf73de7-5618-48db-9e38-37e2b82f372b-kube-api-access-hbwzr\") pod \"ingress-canary-q2rzk\" (UID: \"cdf73de7-5618-48db-9e38-37e2b82f372b\") " pod="openshift-ingress-canary/ingress-canary-q2rzk" Jul 06 23:49:15.417009 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.416919 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-ca\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.417009 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.416957 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-bound-sa-token\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.417009 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.416987 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6d135454-716f-42d9-8f8b-b33fd2228dea-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b49956888-2xnjh\" (UID: \"6d135454-716f-42d9-8f8b-b33fd2228dea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" Jul 06 23:49:15.417321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417014 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-config-volume\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.417321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417043 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.417321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417069 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n9vq\" (UniqueName: \"kubernetes.io/projected/c988e8d2-033f-476d-a7fc-6e7d27acdef8-kube-api-access-8n9vq\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.417321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417099 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-installation-pull-secrets\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.417321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417138 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-tmp-dir\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.417321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417174 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-hub\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.417321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417205 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.417321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417234 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c988e8d2-033f-476d-a7fc-6e7d27acdef8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.417321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417258 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-registry-certificates\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.417321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417282 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbztr\" (UniqueName: \"kubernetes.io/projected/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-kube-api-access-zbztr\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.417691 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417338 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.417691 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417371 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6qls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-kube-api-access-l6qls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.417691 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417416 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-trusted-ca\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.417691 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417439 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.417691 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417463 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/87190854-ff54-4120-bb7d-58bed7aa53ea-klusterlet-config\") pod \"klusterlet-addon-workmgr-d6694ffc5-l4vks\" (UID: \"87190854-ff54-4120-bb7d-58bed7aa53ea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.417691 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.417494 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztxgl\" (UniqueName: \"kubernetes.io/projected/6d135454-716f-42d9-8f8b-b33fd2228dea-kube-api-access-ztxgl\") pod \"managed-serviceaccount-addon-agent-6b49956888-2xnjh\" (UID: \"6d135454-716f-42d9-8f8b-b33fd2228dea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" Jul 06 23:49:15.488664 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.488613 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:15.518635 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.518536 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-ca\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.518635 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.518586 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-bound-sa-token\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.518635 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.518614 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6d135454-716f-42d9-8f8b-b33fd2228dea-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b49956888-2xnjh\" (UID: \"6d135454-716f-42d9-8f8b-b33fd2228dea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" Jul 06 23:49:15.518635 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.518637 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-config-volume\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.518984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.518662 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.518984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.518683 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n9vq\" (UniqueName: \"kubernetes.io/projected/c988e8d2-033f-476d-a7fc-6e7d27acdef8-kube-api-access-8n9vq\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.518984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.518711 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-installation-pull-secrets\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.518984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.518729 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-tmp-dir\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.519550 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519341 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-config-volume\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.519550 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519543 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-hub\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.519757 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519585 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.519757 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519604 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-tmp-dir\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.519757 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519619 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c988e8d2-033f-476d-a7fc-6e7d27acdef8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.519757 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519658 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-registry-certificates\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.519757 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519682 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbztr\" (UniqueName: \"kubernetes.io/projected/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-kube-api-access-zbztr\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.519757 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:15.519702 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:15.519757 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519714 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.519757 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519739 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6qls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-kube-api-access-l6qls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.520194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519774 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-trusted-ca\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.520194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519799 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.520194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519828 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/87190854-ff54-4120-bb7d-58bed7aa53ea-klusterlet-config\") pod \"klusterlet-addon-workmgr-d6694ffc5-l4vks\" (UID: \"87190854-ff54-4120-bb7d-58bed7aa53ea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.520194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519882 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztxgl\" (UniqueName: \"kubernetes.io/projected/6d135454-716f-42d9-8f8b-b33fd2228dea-kube-api-access-ztxgl\") pod \"managed-serviceaccount-addon-agent-6b49956888-2xnjh\" (UID: \"6d135454-716f-42d9-8f8b-b33fd2228dea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" Jul 06 23:49:15.520194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519963 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-image-registry-private-configuration\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.520194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.519988 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0031876a-4b74-4311-917d-8dba25b6df06-ca-trust-extracted\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.520194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.520012 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn8wl\" (UniqueName: \"kubernetes.io/projected/87190854-ff54-4120-bb7d-58bed7aa53ea-kube-api-access-zn8wl\") pod \"klusterlet-addon-workmgr-d6694ffc5-l4vks\" (UID: \"87190854-ff54-4120-bb7d-58bed7aa53ea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.520194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.520038 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87190854-ff54-4120-bb7d-58bed7aa53ea-tmp\") pod \"klusterlet-addon-workmgr-d6694ffc5-l4vks\" (UID: \"87190854-ff54-4120-bb7d-58bed7aa53ea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.520194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.520059 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdf73de7-5618-48db-9e38-37e2b82f372b-cert\") pod \"ingress-canary-q2rzk\" (UID: \"cdf73de7-5618-48db-9e38-37e2b82f372b\") " pod="openshift-ingress-canary/ingress-canary-q2rzk" Jul 06 23:49:15.520194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.520082 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbwzr\" (UniqueName: \"kubernetes.io/projected/cdf73de7-5618-48db-9e38-37e2b82f372b-kube-api-access-hbwzr\") pod \"ingress-canary-q2rzk\" (UID: \"cdf73de7-5618-48db-9e38-37e2b82f372b\") " pod="openshift-ingress-canary/ingress-canary-q2rzk" Jul 06 23:49:15.520725 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.520318 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-registry-certificates\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.521292 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:15.519717 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649bbb5647-76qxk: secret "image-registry-tls" not found Jul 06 23:49:15.521292 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:15.521041 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls podName:0031876a-4b74-4311-917d-8dba25b6df06 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:16.021016643 +0000 UTC m=+36.531177586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls") pod "image-registry-649bbb5647-76qxk" (UID: "0031876a-4b74-4311-917d-8dba25b6df06") : secret "image-registry-tls" not found Jul 06 23:49:15.521292 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:15.520398 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:15.521292 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:15.521240 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls podName:9dfd8b83-8e48-4bcd-8449-bed29a1a166f nodeName:}" failed. No retries permitted until 2026-07-06 23:49:16.021225166 +0000 UTC m=+36.531386102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls") pod "dns-default-s8nvv" (UID: "9dfd8b83-8e48-4bcd-8449-bed29a1a166f") : secret "dns-default-metrics-tls" not found Jul 06 23:49:15.521292 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.521276 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-trusted-ca\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.522530 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.521844 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0031876a-4b74-4311-917d-8dba25b6df06-ca-trust-extracted\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.522530 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.522083 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c988e8d2-033f-476d-a7fc-6e7d27acdef8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.522530 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.522486 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87190854-ff54-4120-bb7d-58bed7aa53ea-tmp\") pod \"klusterlet-addon-workmgr-d6694ffc5-l4vks\" (UID: \"87190854-ff54-4120-bb7d-58bed7aa53ea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.524250 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.524229 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.524356 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.524271 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-ca\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.524432 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.524409 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-installation-pull-secrets\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.524590 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.524549 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/87190854-ff54-4120-bb7d-58bed7aa53ea-klusterlet-config\") pod \"klusterlet-addon-workmgr-d6694ffc5-l4vks\" (UID: \"87190854-ff54-4120-bb7d-58bed7aa53ea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.524824 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.524798 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6d135454-716f-42d9-8f8b-b33fd2228dea-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b49956888-2xnjh\" (UID: \"6d135454-716f-42d9-8f8b-b33fd2228dea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" Jul 06 23:49:15.525620 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.525574 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-hub\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.526076 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.526053 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-image-registry-private-configuration\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.526316 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.526295 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c988e8d2-033f-476d-a7fc-6e7d27acdef8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.527114 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.527087 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdf73de7-5618-48db-9e38-37e2b82f372b-cert\") pod \"ingress-canary-q2rzk\" (UID: \"cdf73de7-5618-48db-9e38-37e2b82f372b\") " pod="openshift-ingress-canary/ingress-canary-q2rzk" Jul 06 23:49:15.529169 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.528756 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-bound-sa-token\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.530090 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.530044 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbwzr\" (UniqueName: \"kubernetes.io/projected/cdf73de7-5618-48db-9e38-37e2b82f372b-kube-api-access-hbwzr\") pod \"ingress-canary-q2rzk\" (UID: \"cdf73de7-5618-48db-9e38-37e2b82f372b\") " pod="openshift-ingress-canary/ingress-canary-q2rzk" Jul 06 23:49:15.530334 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.530314 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n9vq\" (UniqueName: \"kubernetes.io/projected/c988e8d2-033f-476d-a7fc-6e7d27acdef8-kube-api-access-8n9vq\") pod \"cluster-proxy-proxy-agent-6d9f45c469-j7ln8\" (UID: \"c988e8d2-033f-476d-a7fc-6e7d27acdef8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.531033 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.530995 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn8wl\" (UniqueName: \"kubernetes.io/projected/87190854-ff54-4120-bb7d-58bed7aa53ea-kube-api-access-zn8wl\") pod \"klusterlet-addon-workmgr-d6694ffc5-l4vks\" (UID: \"87190854-ff54-4120-bb7d-58bed7aa53ea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.531961 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.531941 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztxgl\" (UniqueName: \"kubernetes.io/projected/6d135454-716f-42d9-8f8b-b33fd2228dea-kube-api-access-ztxgl\") pod \"managed-serviceaccount-addon-agent-6b49956888-2xnjh\" (UID: \"6d135454-716f-42d9-8f8b-b33fd2228dea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" Jul 06 23:49:15.532331 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.532305 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbztr\" (UniqueName: \"kubernetes.io/projected/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-kube-api-access-zbztr\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:15.532825 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.532804 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6qls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-kube-api-access-l6qls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:15.598310 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.598205 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:15.609025 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.608998 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:15.640478 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.640439 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:49:15.647835 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.647809 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" Jul 06 23:49:15.672177 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:15.672142 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q2rzk" Jul 06 23:49:16.024895 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.024834 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:16.025073 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.024925 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:16.025073 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:16.024977 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:16.025073 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:16.024994 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649bbb5647-76qxk: secret "image-registry-tls" not found Jul 06 23:49:16.025073 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:16.025032 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:16.025073 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:16.025058 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls podName:0031876a-4b74-4311-917d-8dba25b6df06 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:17.025040538 +0000 UTC m=+37.535201486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls") pod "image-registry-649bbb5647-76qxk" (UID: "0031876a-4b74-4311-917d-8dba25b6df06") : secret "image-registry-tls" not found Jul 06 23:49:16.025253 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:16.025096 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls podName:9dfd8b83-8e48-4bcd-8449-bed29a1a166f nodeName:}" failed. No retries permitted until 2026-07-06 23:49:17.025083526 +0000 UTC m=+37.535244461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls") pod "dns-default-s8nvv" (UID: "9dfd8b83-8e48-4bcd-8449-bed29a1a166f") : secret "dns-default-metrics-tls" not found Jul 06 23:49:16.045670 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.045632 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:16.045888 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.045681 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:16.045888 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.045650 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:16.050159 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.050128 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9m58d\"" Jul 06 23:49:16.050282 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.050188 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:49:16.050460 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.050442 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7lfhk\"" Jul 06 23:49:16.050626 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.050608 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:49:16.050743 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.050134 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:49:16.050811 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.050776 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:49:16.411384 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.411208 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q2rzk"] Jul 06 23:49:16.414193 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.414168 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks"] Jul 06 23:49:16.416468 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:49:16.416444 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf73de7_5618_48db_9e38_37e2b82f372b.slice/crio-bb94f975621e333bdee05fe748a9de651b4a39bd9e983f31dcada0956ddb3cd1 WatchSource:0}: Error finding container bb94f975621e333bdee05fe748a9de651b4a39bd9e983f31dcada0956ddb3cd1: Status 404 returned error can't find the container with id bb94f975621e333bdee05fe748a9de651b4a39bd9e983f31dcada0956ddb3cd1 Jul 06 23:49:16.425889 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:49:16.425841 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87190854_ff54_4120_bb7d_58bed7aa53ea.slice/crio-6ab99e1be0b37a90a16ab83241d8794fe1988c525a98ed0331afeb5661fd5e25 WatchSource:0}: Error finding container 6ab99e1be0b37a90a16ab83241d8794fe1988c525a98ed0331afeb5661fd5e25: Status 404 returned error can't find the container with id 6ab99e1be0b37a90a16ab83241d8794fe1988c525a98ed0331afeb5661fd5e25 Jul 06 23:49:16.427055 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.427031 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh"] Jul 06 23:49:16.430340 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:49:16.430319 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d135454_716f_42d9_8f8b_b33fd2228dea.slice/crio-ec276a8455b8d682477950a4c276d0cf84f05044ee77a98a07941a38b97d7587 WatchSource:0}: Error finding container ec276a8455b8d682477950a4c276d0cf84f05044ee77a98a07941a38b97d7587: Status 404 returned error can't find the container with id ec276a8455b8d682477950a4c276d0cf84f05044ee77a98a07941a38b97d7587 Jul 06 23:49:16.433698 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:16.433678 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8"] Jul 06 23:49:16.437210 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:49:16.437187 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc988e8d2_033f_476d_a7fc_6e7d27acdef8.slice/crio-fae6328e17d5565ee2e6a662e6367c18884165624ae42e59323adb4e0fd54408 WatchSource:0}: Error finding container fae6328e17d5565ee2e6a662e6367c18884165624ae42e59323adb4e0fd54408: Status 404 returned error can't find the container with id fae6328e17d5565ee2e6a662e6367c18884165624ae42e59323adb4e0fd54408 Jul 06 23:49:17.034840 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:17.033966 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:17.034840 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:17.034034 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:17.034840 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:17.034252 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:17.034840 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:17.034317 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls podName:9dfd8b83-8e48-4bcd-8449-bed29a1a166f nodeName:}" failed. No retries permitted until 2026-07-06 23:49:19.034298421 +0000 UTC m=+39.544459369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls") pod "dns-default-s8nvv" (UID: "9dfd8b83-8e48-4bcd-8449-bed29a1a166f") : secret "dns-default-metrics-tls" not found Jul 06 23:49:17.034840 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:17.034734 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:17.034840 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:17.034753 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649bbb5647-76qxk: secret "image-registry-tls" not found Jul 06 23:49:17.034840 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:17.034806 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls podName:0031876a-4b74-4311-917d-8dba25b6df06 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:19.034785627 +0000 UTC m=+39.544946594 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls") pod "image-registry-649bbb5647-76qxk" (UID: "0031876a-4b74-4311-917d-8dba25b6df06") : secret "image-registry-tls" not found Jul 06 23:49:17.351819 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:17.351687 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" event={"ID":"6d135454-716f-42d9-8f8b-b33fd2228dea","Type":"ContainerStarted","Data":"ec276a8455b8d682477950a4c276d0cf84f05044ee77a98a07941a38b97d7587"} Jul 06 23:49:17.360665 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:17.360488 2558 generic.go:356] "Generic (PLEG): container finished" podID="4ebb3c3f-eda3-4c28-987b-11627a176748" containerID="58438b3d44385ca7b401edd7bd9e4902fbcb6b27714c89b5329d662f604f371a" exitCode=0 Jul 06 23:49:17.360665 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:17.360598 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" event={"ID":"4ebb3c3f-eda3-4c28-987b-11627a176748","Type":"ContainerDied","Data":"58438b3d44385ca7b401edd7bd9e4902fbcb6b27714c89b5329d662f604f371a"} Jul 06 23:49:17.364957 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:17.364879 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" event={"ID":"c988e8d2-033f-476d-a7fc-6e7d27acdef8","Type":"ContainerStarted","Data":"fae6328e17d5565ee2e6a662e6367c18884165624ae42e59323adb4e0fd54408"} Jul 06 23:49:17.369410 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:17.369376 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" event={"ID":"87190854-ff54-4120-bb7d-58bed7aa53ea","Type":"ContainerStarted","Data":"6ab99e1be0b37a90a16ab83241d8794fe1988c525a98ed0331afeb5661fd5e25"} Jul 06 23:49:17.381542 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:17.380217 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q2rzk" event={"ID":"cdf73de7-5618-48db-9e38-37e2b82f372b","Type":"ContainerStarted","Data":"bb94f975621e333bdee05fe748a9de651b4a39bd9e983f31dcada0956ddb3cd1"} Jul 06 23:49:18.121897 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:18.121785 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:18.218430 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:18.218387 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:18.324468 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:18.324405 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:18.392305 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:18.392216 2558 generic.go:356] "Generic (PLEG): container finished" podID="4ebb3c3f-eda3-4c28-987b-11627a176748" containerID="001e336703ee2904d3ffe0d3d7aeaac73bacdc18ef88fa7f30a8e1c921c39ff7" exitCode=0 Jul 06 23:49:18.392305 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:18.392299 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" event={"ID":"4ebb3c3f-eda3-4c28-987b-11627a176748","Type":"ContainerDied","Data":"001e336703ee2904d3ffe0d3d7aeaac73bacdc18ef88fa7f30a8e1c921c39ff7"} Jul 06 23:49:19.009099 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:19.008836 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:19.058102 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:19.057212 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:19.058102 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:19.057279 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:19.058102 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:19.057423 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:19.058102 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:19.057488 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls podName:9dfd8b83-8e48-4bcd-8449-bed29a1a166f nodeName:}" failed. No retries permitted until 2026-07-06 23:49:23.057467802 +0000 UTC m=+43.567628760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls") pod "dns-default-s8nvv" (UID: "9dfd8b83-8e48-4bcd-8449-bed29a1a166f") : secret "dns-default-metrics-tls" not found Jul 06 23:49:19.058102 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:19.057965 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:19.058102 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:19.057983 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649bbb5647-76qxk: secret "image-registry-tls" not found Jul 06 23:49:19.058102 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:19.058034 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls podName:0031876a-4b74-4311-917d-8dba25b6df06 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:23.058016941 +0000 UTC m=+43.568177881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls") pod "image-registry-649bbb5647-76qxk" (UID: "0031876a-4b74-4311-917d-8dba25b6df06") : secret "image-registry-tls" not found Jul 06 23:49:19.109605 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:19.109450 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:19.399163 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:19.399088 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" event={"ID":"4ebb3c3f-eda3-4c28-987b-11627a176748","Type":"ContainerStarted","Data":"d29a57cfebcc532903ea0b44833e2d9a97bfd9b9a34d82da0d04aa2ad5c81ba9"} Jul 06 23:49:19.423346 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:19.423289 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tw7t9" podStartSLOduration=8.796189881 podStartE2EDuration="39.423266724s" totalImagesPullingTime="30.627076843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:19.421560004 +0000 UTC m=+39.931720975" watchObservedRunningTime="2026-07-06 23:49:19.423266724 +0000 UTC m=+39.933427684" Jul 06 23:49:19.544704 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:19.544669 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:19.609980 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:19.609924 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:20.070732 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:20.070694 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:20.126439 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:20.126406 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:20.408641 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:20.408500 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:20.512118 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:20.512069 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:23.089230 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:23.089191 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:23.089230 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:23.089235 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:23.089825 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:23.089359 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:23.089825 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:23.089372 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:23.089825 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:23.089400 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649bbb5647-76qxk: secret "image-registry-tls" not found Jul 06 23:49:23.089825 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:23.089427 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls podName:9dfd8b83-8e48-4bcd-8449-bed29a1a166f nodeName:}" failed. No retries permitted until 2026-07-06 23:49:31.08940792 +0000 UTC m=+51.599568878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls") pod "dns-default-s8nvv" (UID: "9dfd8b83-8e48-4bcd-8449-bed29a1a166f") : secret "dns-default-metrics-tls" not found Jul 06 23:49:23.089825 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:23.089456 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls podName:0031876a-4b74-4311-917d-8dba25b6df06 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:31.08943713 +0000 UTC m=+51.599598067 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls") pod "image-registry-649bbb5647-76qxk" (UID: "0031876a-4b74-4311-917d-8dba25b6df06") : secret "image-registry-tls" not found Jul 06 23:49:25.416566 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.416478 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" event={"ID":"c988e8d2-033f-476d-a7fc-6e7d27acdef8","Type":"ContainerStarted","Data":"045743c97cfd20711b3d48a335ae1009003a8d7d6645286633a12b8a5965eaba"} Jul 06 23:49:25.416566 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.416526 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" event={"ID":"c988e8d2-033f-476d-a7fc-6e7d27acdef8","Type":"ContainerStarted","Data":"9d303ade823dd9726dd71c7283be88b2fb333b5e7a5186a99a574b92f07fcebb"} Jul 06 23:49:25.416566 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.416547 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" event={"ID":"c988e8d2-033f-476d-a7fc-6e7d27acdef8","Type":"ContainerStarted","Data":"21079fbdb56e1c1acfeab33b9113e4729fbcbcaea5533e7bc9237ae9ff9f2365"} Jul 06 23:49:25.417992 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.417965 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" event={"ID":"87190854-ff54-4120-bb7d-58bed7aa53ea","Type":"ContainerStarted","Data":"fd618143a66cf16d7ef50333724526dfbcd0c30b45c67ec0a9054bed323ba820"} Jul 06 23:49:25.418117 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.418037 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:25.419419 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.419391 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q2rzk" event={"ID":"cdf73de7-5618-48db-9e38-37e2b82f372b","Type":"ContainerStarted","Data":"dbe4c4f2d87d8244c223e0bb69034d79aa6fd3dda2263478e0531a3864afc001"} Jul 06 23:49:25.420025 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.420005 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" Jul 06 23:49:25.420803 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.420783 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" event={"ID":"6d135454-716f-42d9-8f8b-b33fd2228dea","Type":"ContainerStarted","Data":"5bef30bd0a78710cac7450c9d63d99488bfb9adc64086f99a1303f976b7673ec"} Jul 06 23:49:25.435134 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.435102 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:25.444053 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.444003 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" podStartSLOduration=29.469746417 podStartE2EDuration="37.443988765s" totalImagesPullingTime="7.974242348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:25.442136302 +0000 UTC m=+45.952297280" watchObservedRunningTime="2026-07-06 23:49:25.443988765 +0000 UTC m=+45.954149722" Jul 06 23:49:25.458477 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.458433 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6694ffc5-l4vks" podStartSLOduration=29.488679185 podStartE2EDuration="37.458416685s" totalImagesPullingTime="7.9697375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:25.457884459 +0000 UTC m=+45.968045431" watchObservedRunningTime="2026-07-06 23:49:25.458416685 +0000 UTC m=+45.968577619" Jul 06 23:49:25.475094 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.475042 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q2rzk" podStartSLOduration=2.4957502 podStartE2EDuration="10.475025238s" totalImagesPullingTime="7.979275038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:25.47430151 +0000 UTC m=+45.984462468" watchObservedRunningTime="2026-07-06 23:49:25.475025238 +0000 UTC m=+45.985186194" Jul 06 23:49:25.491554 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.491489 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b49956888-2xnjh" podStartSLOduration=29.527762861 podStartE2EDuration="37.49147187s" totalImagesPullingTime="7.963709009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:25.490885486 +0000 UTC m=+46.001046443" watchObservedRunningTime="2026-07-06 23:49:25.49147187 +0000 UTC m=+46.001632864" Jul 06 23:49:25.536025 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.535979 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:25.890173 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.890139 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:25.993523 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:25.993491 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:26.082141 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:26.082107 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:26.906590 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:26.906558 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:27.005001 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:27.004966 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:27.105389 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:27.105349 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:31.025674 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:31.025635 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:31.090329 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:31.090287 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:31.160347 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:31.160311 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:31.160493 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:31.160361 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:31.160493 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:31.160478 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:31.160493 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:31.160495 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649bbb5647-76qxk: secret "image-registry-tls" not found Jul 06 23:49:31.160608 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:31.160516 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:31.160608 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:31.160559 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls podName:0031876a-4b74-4311-917d-8dba25b6df06 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:47.160542774 +0000 UTC m=+67.670703711 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls") pod "image-registry-649bbb5647-76qxk" (UID: "0031876a-4b74-4311-917d-8dba25b6df06") : secret "image-registry-tls" not found Jul 06 23:49:31.160608 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:31.160573 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls podName:9dfd8b83-8e48-4bcd-8449-bed29a1a166f nodeName:}" failed. No retries permitted until 2026-07-06 23:49:47.16056725 +0000 UTC m=+67.670728185 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls") pod "dns-default-s8nvv" (UID: "9dfd8b83-8e48-4bcd-8449-bed29a1a166f") : secret "dns-default-metrics-tls" not found Jul 06 23:49:34.869789 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:34.869748 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:34.923060 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:34.923026 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:35.166889 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:35.166768 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:35.248240 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:35.248200 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:36.865414 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:36.865382 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:36.940896 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:36.940834 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:37.733095 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:37.733044 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:37.829525 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:37.829477 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:42.021696 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:42.021651 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:42.124110 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:42.124076 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:42.222897 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:42.222844 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:42.344618 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:42.344539 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z7v2" Jul 06 23:49:42.508045 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:42.508005 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:42.605218 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:42.605122 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:44.200543 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.200505 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:44.295756 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.295711 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:44.860430 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.860378 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:49:44.860430 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.860440 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:44.863078 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.863058 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:49:44.863185 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.863164 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:49:44.871471 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:44.871448 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 23:49:44.871546 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:44.871535 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs podName:c5c374b5-74a1-41b6-826c-288b284cc627 nodeName:}" failed. No retries permitted until 2026-07-06 23:50:48.87150681 +0000 UTC m=+129.381667746 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs") pod "network-metrics-daemon-c5zp8" (UID: "c5c374b5-74a1-41b6-826c-288b284cc627") : secret "metrics-daemon-secret" not found Jul 06 23:49:44.875512 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.875479 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3dee8e4b-8ebb-470a-aaac-a610ba92c6dd-original-pull-secret\") pod \"global-pull-secret-syncer-8dl25\" (UID: \"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd\") " pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:44.961425 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.961383 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbsxc\" (UniqueName: \"kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc\") pod \"network-check-target-zdbst\" (UID: \"7de4f318-5759-4f4f-870e-01b72f5f3e90\") " pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:44.964464 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.964445 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:49:44.974110 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.974086 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:49:44.984765 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:44.984734 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbsxc\" (UniqueName: \"kubernetes.io/projected/7de4f318-5759-4f4f-870e-01b72f5f3e90-kube-api-access-rbsxc\") pod \"network-check-target-zdbst\" (UID: \"7de4f318-5759-4f4f-870e-01b72f5f3e90\") " pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:45.123827 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.123739 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:45.160397 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.160367 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7lfhk\"" Jul 06 23:49:45.168006 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.167989 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:45.170029 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.170011 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8dl25" Jul 06 23:49:45.303128 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.303079 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:45.324255 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.324226 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8dl25"] Jul 06 23:49:45.327212 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:49:45.327185 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dee8e4b_8ebb_470a_aaac_a610ba92c6dd.slice/crio-7135ff79c5e61fb6a216c76569814e68b5cfb72b814e645444161a8df928ac0e WatchSource:0}: Error finding container 7135ff79c5e61fb6a216c76569814e68b5cfb72b814e645444161a8df928ac0e: Status 404 returned error can't find the container with id 7135ff79c5e61fb6a216c76569814e68b5cfb72b814e645444161a8df928ac0e Jul 06 23:49:45.333733 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.333709 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zdbst"] Jul 06 23:49:45.336661 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:49:45.336638 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de4f318_5759_4f4f_870e_01b72f5f3e90.slice/crio-eaf3a238cae548285162934b3cc87f12e67efa5859295854cb75665f500c8610 WatchSource:0}: Error finding container eaf3a238cae548285162934b3cc87f12e67efa5859295854cb75665f500c8610: Status 404 returned error can't find the container with id eaf3a238cae548285162934b3cc87f12e67efa5859295854cb75665f500c8610 Jul 06 23:49:45.397418 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.397317 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:45.477337 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.477303 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8dl25" event={"ID":"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd","Type":"ContainerStarted","Data":"7135ff79c5e61fb6a216c76569814e68b5cfb72b814e645444161a8df928ac0e"} Jul 06 23:49:45.478488 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.478465 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zdbst" event={"ID":"7de4f318-5759-4f4f-870e-01b72f5f3e90","Type":"ContainerStarted","Data":"eaf3a238cae548285162934b3cc87f12e67efa5859295854cb75665f500c8610"} Jul 06 23:49:45.531123 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.531097 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-c44w6"] Jul 06 23:49:45.534013 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.533994 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.537190 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.537165 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 23:49:45.537313 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.537167 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 23:49:45.537313 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.537216 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 23:49:45.537313 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.537235 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lr2cs\"" Jul 06 23:49:45.537313 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.537201 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 23:49:45.551440 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.551411 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c44w6"] Jul 06 23:49:45.565405 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.565377 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.565549 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.565416 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42e0dccb-3536-4e4c-8aa7-fe4c34741997-data-volume\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.565549 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.565438 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42e0dccb-3536-4e4c-8aa7-fe4c34741997-crio-socket\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.565549 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.565506 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.565650 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.565557 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhdzp\" (UniqueName: \"kubernetes.io/projected/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-api-access-hhdzp\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.581630 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.581595 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:45.643509 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.643470 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:45.666638 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.666560 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.666638 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.666602 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42e0dccb-3536-4e4c-8aa7-fe4c34741997-data-volume\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.666638 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.666630 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42e0dccb-3536-4e4c-8aa7-fe4c34741997-crio-socket\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.666838 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.666670 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.666838 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.666698 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhdzp\" (UniqueName: \"kubernetes.io/projected/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-api-access-hhdzp\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.666838 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:45.666731 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:45.666838 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.666789 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42e0dccb-3536-4e4c-8aa7-fe4c34741997-crio-socket\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.666838 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:45.666797 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:45.667050 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:45.666806 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:46.166787967 +0000 UTC m=+66.676948920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:45.667050 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:45.666914 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:46.166895848 +0000 UTC m=+66.677056782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : configmap "kube-rbac-proxy" not found Jul 06 23:49:45.667050 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.667004 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42e0dccb-3536-4e4c-8aa7-fe4c34741997-data-volume\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.676242 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.676219 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhdzp\" (UniqueName: \"kubernetes.io/projected/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-api-access-hhdzp\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:45.845399 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.845332 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:45.906682 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:45.906643 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:46.171965 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:46.171924 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:46.172168 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:46.172015 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:46.172168 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:46.172149 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:46.172291 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:46.172224 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:47.172204988 +0000 UTC m=+67.682365925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : configmap "kube-rbac-proxy" not found Jul 06 23:49:46.172473 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:46.172447 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:46.172602 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:46.172520 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:47.172501872 +0000 UTC m=+67.682662819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:46.195025 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:46.194990 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:46.253120 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:46.253022 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:46.458510 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:46.458421 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:46.563371 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:46.563339 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:47.180841 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:47.180954 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:47.180990 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:47.181009 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:47.181037 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:47.181070 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:49.181046697 +0000 UTC m=+69.691207632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:47.181138 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:47.181187 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls podName:9dfd8b83-8e48-4bcd-8449-bed29a1a166f nodeName:}" failed. No retries permitted until 2026-07-06 23:50:19.18117129 +0000 UTC m=+99.691332231 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls") pod "dns-default-s8nvv" (UID: "9dfd8b83-8e48-4bcd-8449-bed29a1a166f") : secret "dns-default-metrics-tls" not found Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:47.181233 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:47.181264 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:49.181254287 +0000 UTC m=+69.691415242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : configmap "kube-rbac-proxy" not found Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:47.181324 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:47.181335 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649bbb5647-76qxk: secret "image-registry-tls" not found Jul 06 23:49:47.181392 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:47.181365 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls podName:0031876a-4b74-4311-917d-8dba25b6df06 nodeName:}" failed. No retries permitted until 2026-07-06 23:50:19.181355052 +0000 UTC m=+99.691515992 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls") pod "image-registry-649bbb5647-76qxk" (UID: "0031876a-4b74-4311-917d-8dba25b6df06") : secret "image-registry-tls" not found Jul 06 23:49:47.998679 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:47.998636 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:48.092819 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:48.092774 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:48.169671 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:48.169625 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:48.543170 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:48.543138 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wxmxp_4423aa72-95c7-4610-9d94-76b3f1c2b76e/dns-node-resolver/0.log" Jul 06 23:49:49.146444 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:49.146416 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-brsvk_9ff9b33f-ec62-4d90-80dc-745c2ffa9498/node-ca/0.log" Jul 06 23:49:49.198518 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:49.198483 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:49.198725 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:49.198544 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:49.198725 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:49.198647 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:49.198725 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:49.198722 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:53.19870569 +0000 UTC m=+73.708866629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:49.198879 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:49.198649 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:49.198879 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:49.198828 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:53.198813956 +0000 UTC m=+73.708974892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : configmap "kube-rbac-proxy" not found Jul 06 23:49:49.226111 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:49.226067 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:50.341550 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:50.341522 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q2rzk_cdf73de7-5618-48db-9e38-37e2b82f372b/serve-healthcheck-canary/0.log" Jul 06 23:49:50.498201 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:50.498163 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zdbst" event={"ID":"7de4f318-5759-4f4f-870e-01b72f5f3e90","Type":"ContainerStarted","Data":"ff1fc9e8e7775ae6de3bae2c8e78b4ec37b0503f4f2ccbb4dc8622a334b565a6"} Jul 06 23:49:50.498384 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:50.498222 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:49:50.499654 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:50.499630 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8dl25" event={"ID":"3dee8e4b-8ebb-470a-aaac-a610ba92c6dd","Type":"ContainerStarted","Data":"86dbe20d64a09cabda72e71195f64338993a86fb1dfc99968ec405f457cb71af"} Jul 06 23:49:50.515913 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:50.515837 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zdbst" podStartSLOduration=65.943766834 podStartE2EDuration="1m10.515824868s" totalImagesPullingTime="4.572058034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:50.515555782 +0000 UTC m=+71.025716758" watchObservedRunningTime="2026-07-06 23:49:50.515824868 +0000 UTC m=+71.025985803" Jul 06 23:49:50.533136 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:50.533083 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8dl25" podStartSLOduration=65.948071942 podStartE2EDuration="1m10.533065833s" totalImagesPullingTime="4.584993891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:50.532695439 +0000 UTC m=+71.042856395" watchObservedRunningTime="2026-07-06 23:49:50.533065833 +0000 UTC m=+71.043226789" Jul 06 23:49:52.443944 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:52.443906 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:52.817094 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:52.816989 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:53.225913 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:53.225874 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:53.226099 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:53.225935 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:49:53.226099 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:53.226035 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:53.226099 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:53.226037 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:53.226099 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:53.226088 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:50:01.226074578 +0000 UTC m=+81.736235513 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : configmap "kube-rbac-proxy" not found Jul 06 23:49:53.226099 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:49:53.226101 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:50:01.226095382 +0000 UTC m=+81.736256318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:53.235916 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:53.235883 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:53.335436 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:53.335399 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:53.439002 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:53.438964 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:53.670389 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:53.670355 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:54.592251 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:54.592192 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:54.986870 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:54.986820 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:55.185727 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:55.185684 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:55.391528 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:55.391421 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:55.792401 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:55.792364 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:56.396921 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:56.396881 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:56.712091 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:56.712057 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:56.812468 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:56.812428 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:56.870193 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:56.870127 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:56.987556 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:56.987460 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:57.634100 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:57.634051 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:57.749185 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:57.749152 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:58.017172 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:58.017139 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:58.223633 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:58.223603 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:58.408911 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:58.408799 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:58.594271 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:58.594234 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:58.694565 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:58.694537 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:58.796060 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:58.796028 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:58.949255 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:58.949168 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:59.205527 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:59.205422 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:59.268339 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:59.268309 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:59.324804 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:59.324766 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:59.730124 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:59.730090 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:59.831724 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:59.831688 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:59.931985 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:49:59.931950 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:00.032584 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:00.032471 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:00.199356 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:00.199317 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:00.343200 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:00.343093 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:00.546232 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:00.546196 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:00.648451 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:00.648358 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:01.277094 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:01.277059 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:01.294218 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:01.294180 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:50:01.294319 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:01.294237 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:50:01.294363 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:50:01.294345 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:50:01.294425 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:50:01.294414 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls podName:42e0dccb-3536-4e4c-8aa7-fe4c34741997 nodeName:}" failed. No retries permitted until 2026-07-06 23:50:17.294399279 +0000 UTC m=+97.804560214 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c44w6" (UID: "42e0dccb-3536-4e4c-8aa7-fe4c34741997") : secret "insights-runtime-extractor-tls" not found Jul 06 23:50:01.294719 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:01.294703 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42e0dccb-3536-4e4c-8aa7-fe4c34741997-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:50:01.404696 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:01.404657 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:01.602708 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:01.602618 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:01.961342 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:01.961304 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:02.022041 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:02.022009 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:02.199103 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:02.199068 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:02.592412 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:02.592380 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:03.395916 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:03.395863 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:03.598820 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:03.598782 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:03.793523 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:03.793483 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:03.995660 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:03.995626 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:08.365596 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:08.365559 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:08.434193 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:08.434157 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:10.215874 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:10.215820 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:10.302377 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:10.302346 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:10.401915 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:10.401882 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:10.505969 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:10.505883 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:10.949671 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:10.949626 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:11.019218 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:11.019172 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:11.117530 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:11.117494 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:11.295010 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:11.294919 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:11.413713 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:11.413683 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:11.751465 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:11.751423 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:11.851455 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:11.851416 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:15.368464 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:15.368427 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:15.468844 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:15.468806 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:16.227454 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:16.227413 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:16.285933 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:16.285891 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:17.171758 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:17.171722 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:17.250266 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:17.250223 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:17.318298 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:17.318266 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:50:17.320799 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:17.320778 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42e0dccb-3536-4e4c-8aa7-fe4c34741997-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c44w6\" (UID: \"42e0dccb-3536-4e4c-8aa7-fe4c34741997\") " pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:50:17.343411 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:17.343366 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c44w6" Jul 06 23:50:17.476654 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:17.476621 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c44w6"] Jul 06 23:50:17.479538 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:50:17.479504 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e0dccb_3536_4e4c_8aa7_fe4c34741997.slice/crio-b442f3e7e3bccdcbb0dbe1763707f69903fe370fc50ad514a0c864f33fbc7edc WatchSource:0}: Error finding container b442f3e7e3bccdcbb0dbe1763707f69903fe370fc50ad514a0c864f33fbc7edc: Status 404 returned error can't find the container with id b442f3e7e3bccdcbb0dbe1763707f69903fe370fc50ad514a0c864f33fbc7edc Jul 06 23:50:17.555692 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:17.555660 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:17.584810 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:17.584776 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c44w6" event={"ID":"42e0dccb-3536-4e4c-8aa7-fe4c34741997","Type":"ContainerStarted","Data":"b442f3e7e3bccdcbb0dbe1763707f69903fe370fc50ad514a0c864f33fbc7edc"} Jul 06 23:50:17.656360 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:17.656316 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:17.719358 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:17.719262 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:19.232657 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:19.232614 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:50:19.233109 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:19.232672 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:50:19.235332 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:19.235297 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dfd8b83-8e48-4bcd-8449-bed29a1a166f-metrics-tls\") pod \"dns-default-s8nvv\" (UID: \"9dfd8b83-8e48-4bcd-8449-bed29a1a166f\") " pod="openshift-dns/dns-default-s8nvv" Jul 06 23:50:19.235596 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:19.235575 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"image-registry-649bbb5647-76qxk\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:50:19.267379 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:19.267342 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5vhf7\"" Jul 06 23:50:19.275076 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:19.275042 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:50:19.282427 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:19.282401 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p99x7\"" Jul 06 23:50:19.290200 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:19.290174 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s8nvv" Jul 06 23:50:19.967312 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:19.967274 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:20.024424 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:20.024383 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:21.446766 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:21.446724 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s8nvv"] Jul 06 23:50:21.449821 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:50:21.449787 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dfd8b83_8e48_4bcd_8449_bed29a1a166f.slice/crio-6563f721a4b8fed7fbc05ba67319677e156df3ba21eb029fc3d64f8dfb0ff3ed WatchSource:0}: Error finding container 6563f721a4b8fed7fbc05ba67319677e156df3ba21eb029fc3d64f8dfb0ff3ed: Status 404 returned error can't find the container with id 6563f721a4b8fed7fbc05ba67319677e156df3ba21eb029fc3d64f8dfb0ff3ed Jul 06 23:50:21.450272 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:21.450253 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-649bbb5647-76qxk"] Jul 06 23:50:21.505076 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:21.505042 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zdbst" Jul 06 23:50:21.603156 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:21.603076 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s8nvv" event={"ID":"9dfd8b83-8e48-4bcd-8449-bed29a1a166f","Type":"ContainerStarted","Data":"6563f721a4b8fed7fbc05ba67319677e156df3ba21eb029fc3d64f8dfb0ff3ed"} Jul 06 23:50:21.616514 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:50:21.616485 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0031876a_4b74_4311_917d_8dba25b6df06.slice/crio-a95feeac9183dca180cfc9116a78fb7df06c4519076801d7cca7a2d8f08c9d72 WatchSource:0}: Error finding container a95feeac9183dca180cfc9116a78fb7df06c4519076801d7cca7a2d8f08c9d72: Status 404 returned error can't find the container with id a95feeac9183dca180cfc9116a78fb7df06c4519076801d7cca7a2d8f08c9d72 Jul 06 23:50:22.608931 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:22.608895 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c44w6" event={"ID":"42e0dccb-3536-4e4c-8aa7-fe4c34741997","Type":"ContainerStarted","Data":"f368cd5e278a39049868bc8c76507f827c6940e4f58e3ca0ad96f69a330cc3e3"} Jul 06 23:50:22.611320 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:22.611285 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" event={"ID":"0031876a-4b74-4311-917d-8dba25b6df06","Type":"ContainerStarted","Data":"7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016"} Jul 06 23:50:22.611448 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:22.611332 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" event={"ID":"0031876a-4b74-4311-917d-8dba25b6df06","Type":"ContainerStarted","Data":"a95feeac9183dca180cfc9116a78fb7df06c4519076801d7cca7a2d8f08c9d72"} Jul 06 23:50:22.611448 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:22.611436 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:50:23.566127 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:23.566068 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:23.616697 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:23.616602 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c44w6" event={"ID":"42e0dccb-3536-4e4c-8aa7-fe4c34741997","Type":"ContainerStarted","Data":"96128c9e478d194295e033506aeed18a5bb97ac47442a97890661ddefd55de3e"} Jul 06 23:50:23.665698 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:23.665638 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:24.624181 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:24.624136 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s8nvv" event={"ID":"9dfd8b83-8e48-4bcd-8449-bed29a1a166f","Type":"ContainerStarted","Data":"e03d9320192ed638e9d82c4dbdd2e977f16e7ef003c20b81085cf4174d7e7743"} Jul 06 23:50:24.624623 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:24.624206 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s8nvv" event={"ID":"9dfd8b83-8e48-4bcd-8449-bed29a1a166f","Type":"ContainerStarted","Data":"266d61dc5715d546f72e1c98225fe60dcc9aa7a0b6f945ad0232ce60554fd4c2"} Jul 06 23:50:24.624623 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:24.624286 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-s8nvv" Jul 06 23:50:24.645552 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:24.645491 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" podStartSLOduration=80.645474338 podStartE2EDuration="1m20.645474338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:22.641534541 +0000 UTC m=+103.151695499" watchObservedRunningTime="2026-07-06 23:50:24.645474338 +0000 UTC m=+105.155635294" Jul 06 23:50:24.646074 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:24.646035 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-s8nvv" podStartSLOduration=67.55504663 podStartE2EDuration="1m9.646024901s" totalImagesPullingTime="2.090978271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:24.64414893 +0000 UTC m=+105.154309879" watchObservedRunningTime="2026-07-06 23:50:24.646024901 +0000 UTC m=+105.156185857" Jul 06 23:50:26.156962 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:26.156928 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:26.247344 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:26.247301 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:26.634090 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:26.634058 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c44w6" event={"ID":"42e0dccb-3536-4e4c-8aa7-fe4c34741997","Type":"ContainerStarted","Data":"df19dcbeafead4a2b3af37dd35a3b9b29c09765db0a042fc7ee558e736d172f9"} Jul 06 23:50:26.653511 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:26.653459 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-c44w6" podStartSLOduration=33.290190425 podStartE2EDuration="41.653441105s" totalImagesPullingTime="8.36325068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:49:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:26.652563499 +0000 UTC m=+107.162724458" watchObservedRunningTime="2026-07-06 23:50:26.653441105 +0000 UTC m=+107.163602062" Jul 06 23:50:27.775971 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:27.775926 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:30.367374 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:30.367337 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:30.431762 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:30.431729 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:31.610568 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:31.610531 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:31.859797 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:31.859764 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:31.917642 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:31.917559 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:32.038408 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:32.038374 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-649bbb5647-76qxk"] Jul 06 23:50:32.128223 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:32.128184 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:32.225604 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:32.225562 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:33.312661 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.312619 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:33.519267 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.519228 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:33.573368 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.573295 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bnlwl"] Jul 06 23:50:33.576380 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.576361 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.579780 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.579759 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 23:50:33.580791 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.580764 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 23:50:33.580791 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.580779 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 23:50:33.580999 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.580798 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 23:50:33.580999 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.580768 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 23:50:33.580999 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.580963 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j6ctg\"" Jul 06 23:50:33.580999 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.580966 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 23:50:33.640475 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.640435 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:33.725321 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.725285 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:33.746633 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.746599 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-wtmp\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.746769 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.746646 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-accelerators-collector-config\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.746769 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.746705 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbw7b\" (UniqueName: \"kubernetes.io/projected/87b29f4b-efd0-490b-9611-1f1d01633846-kube-api-access-cbw7b\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.746952 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.746788 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-textfile\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.746952 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.746820 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87b29f4b-efd0-490b-9611-1f1d01633846-root\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.746952 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.746876 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87b29f4b-efd0-490b-9611-1f1d01633846-metrics-client-ca\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.746952 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.746907 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-tls\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.746952 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.746933 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87b29f4b-efd0-490b-9611-1f1d01633846-sys\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.747131 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.746958 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.825505 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.825411 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:33.847458 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847428 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbw7b\" (UniqueName: \"kubernetes.io/projected/87b29f4b-efd0-490b-9611-1f1d01633846-kube-api-access-cbw7b\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.847562 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847491 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-textfile\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.847562 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847515 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87b29f4b-efd0-490b-9611-1f1d01633846-root\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.847562 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847533 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87b29f4b-efd0-490b-9611-1f1d01633846-metrics-client-ca\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.847562 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847551 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-tls\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.847788 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847573 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87b29f4b-efd0-490b-9611-1f1d01633846-sys\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.847788 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847597 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.847788 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847622 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87b29f4b-efd0-490b-9611-1f1d01633846-root\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.847788 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847636 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-wtmp\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.847788 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847682 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-accelerators-collector-config\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.847788 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847763 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-wtmp\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.848093 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.847811 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87b29f4b-efd0-490b-9611-1f1d01633846-sys\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.848093 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.848014 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-textfile\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.848314 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.848291 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-accelerators-collector-config\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.848490 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.848314 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87b29f4b-efd0-490b-9611-1f1d01633846-metrics-client-ca\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.850216 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.850194 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.850317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.850274 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87b29f4b-efd0-490b-9611-1f1d01633846-node-exporter-tls\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.856603 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.856579 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbw7b\" (UniqueName: \"kubernetes.io/projected/87b29f4b-efd0-490b-9611-1f1d01633846-kube-api-access-cbw7b\") pod \"node-exporter-bnlwl\" (UID: \"87b29f4b-efd0-490b-9611-1f1d01633846\") " pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.886424 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.886395 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bnlwl" Jul 06 23:50:33.898591 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:50:33.898559 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87b29f4b_efd0_490b_9611_1f1d01633846.slice/crio-516d8c7befebb31e5e0ee62270d957fca3af46f3806c555140f99b2b2a83ed8f WatchSource:0}: Error finding container 516d8c7befebb31e5e0ee62270d957fca3af46f3806c555140f99b2b2a83ed8f: Status 404 returned error can't find the container with id 516d8c7befebb31e5e0ee62270d957fca3af46f3806c555140f99b2b2a83ed8f Jul 06 23:50:33.925574 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:33.925525 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:34.025650 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.025612 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:34.130585 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.130488 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:34.196865 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.196813 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:34.263463 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.263422 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:34.433639 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.433608 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:34.546634 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.546588 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:34.631664 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.631626 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-s8nvv" Jul 06 23:50:34.637125 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.637092 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:34.662451 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.662414 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bnlwl" event={"ID":"87b29f4b-efd0-490b-9611-1f1d01633846","Type":"ContainerStarted","Data":"516d8c7befebb31e5e0ee62270d957fca3af46f3806c555140f99b2b2a83ed8f"} Jul 06 23:50:34.769064 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.768974 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:34.865913 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:34.865877 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:35.279496 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:35.279458 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:35.360530 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:35.360433 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:35.542364 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:35.542331 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:35.643947 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:35.643841 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:35.667752 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:35.667714 2558 generic.go:356] "Generic (PLEG): container finished" podID="87b29f4b-efd0-490b-9611-1f1d01633846" containerID="a9328f77290534f9827be2a6986c4ecf3544b85fc8fef52d6e6109d0946b8b50" exitCode=0 Jul 06 23:50:35.667965 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:35.667780 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bnlwl" event={"ID":"87b29f4b-efd0-490b-9611-1f1d01633846","Type":"ContainerDied","Data":"a9328f77290534f9827be2a6986c4ecf3544b85fc8fef52d6e6109d0946b8b50"} Jul 06 23:50:36.590754 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:36.590718 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:36.656486 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:36.656451 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:36.673522 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:36.673480 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bnlwl" event={"ID":"87b29f4b-efd0-490b-9611-1f1d01633846","Type":"ContainerStarted","Data":"5a7e667df98ab085a11aa54a45ae5712d18861b4a141935833f3f1c5cefcb5ef"} Jul 06 23:50:36.673643 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:36.673580 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bnlwl" event={"ID":"87b29f4b-efd0-490b-9611-1f1d01633846","Type":"ContainerStarted","Data":"40a51f6d03911714f33997dfc21aafcae61a308f84be9a0f4229d415f927d271"} Jul 06 23:50:36.695202 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:36.695160 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bnlwl" podStartSLOduration=2.624309105 podStartE2EDuration="3.695145919s" totalImagesPullingTime="997.143458ms" totalInitContainerRuntime="73.693356ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:50:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:36.694514322 +0000 UTC m=+117.204675279" watchObservedRunningTime="2026-07-06 23:50:36.695145919 +0000 UTC m=+117.205306876" Jul 06 23:50:36.754235 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:36.754195 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:36.928609 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:36.928562 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:38.233202 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:38.233164 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:38.337909 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:38.337528 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:38.443917 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:38.443888 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:38.533842 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:38.533755 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:39.014692 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:39.014658 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:39.114867 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:39.114809 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:40.001462 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:40.001431 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:40.220351 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:40.220316 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:40.296663 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:40.296558 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:42.044666 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:42.044635 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:50:43.880405 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:43.880369 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:43.976892 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:43.976833 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:44.317950 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:44.317909 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:44.377211 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:44.377178 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:44.904531 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:44.904490 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:45.005711 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:45.005679 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:48.970154 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:48.970091 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:50:48.972597 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:48.972568 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c374b5-74a1-41b6-826c-288b284cc627-metrics-certs\") pod \"network-metrics-daemon-c5zp8\" (UID: \"c5c374b5-74a1-41b6-826c-288b284cc627\") " pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:50:49.066970 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:49.066941 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9m58d\"" Jul 06 23:50:49.075367 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:49.075341 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c5zp8" Jul 06 23:50:49.206193 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:49.206159 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c5zp8"] Jul 06 23:50:49.209393 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:50:49.209362 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c374b5_74a1_41b6_826c_288b284cc627.slice/crio-2892ebf4ce7aa4486d612ec12967c94116b8b38e5a16a2c801529eac76a8d5c3 WatchSource:0}: Error finding container 2892ebf4ce7aa4486d612ec12967c94116b8b38e5a16a2c801529eac76a8d5c3: Status 404 returned error can't find the container with id 2892ebf4ce7aa4486d612ec12967c94116b8b38e5a16a2c801529eac76a8d5c3 Jul 06 23:50:49.719112 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:49.719072 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c5zp8" event={"ID":"c5c374b5-74a1-41b6-826c-288b284cc627","Type":"ContainerStarted","Data":"2892ebf4ce7aa4486d612ec12967c94116b8b38e5a16a2c801529eac76a8d5c3"} Jul 06 23:50:49.879716 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:49.879624 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:49.981427 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:49.981340 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:50.039243 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:50.039207 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:50.723673 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:50.723630 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c5zp8" event={"ID":"c5c374b5-74a1-41b6-826c-288b284cc627","Type":"ContainerStarted","Data":"69b34670a6034581627ddcb817fd9b79795d72719a5c13f96e3e450c07f098ee"} Jul 06 23:50:50.723832 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:50.723701 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c5zp8" event={"ID":"c5c374b5-74a1-41b6-826c-288b284cc627","Type":"ContainerStarted","Data":"a01a7f086b0fc869357b93a8b55a27c88585b2889a5a31cff3a6dd44eb89f720"} Jul 06 23:50:50.741172 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:50.741122 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c5zp8" podStartSLOduration=129.85883148 podStartE2EDuration="2m10.741106401s" totalImagesPullingTime="882.274921ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:50.739010674 +0000 UTC m=+131.249171650" watchObservedRunningTime="2026-07-06 23:50:50.741106401 +0000 UTC m=+131.251267400" Jul 06 23:50:50.834868 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:50.834808 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:50.892575 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:50.892522 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:55.432724 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:55.432685 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:55.494864 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:55.494815 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:55.592240 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:55.592205 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:55.641959 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:55.641907 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" podUID="c988e8d2-033f-476d-a7fc-6e7d27acdef8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 23:50:57.058616 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.058580 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" podUID="0031876a-4b74-4311-917d-8dba25b6df06" containerName="registry" containerID="cri-o://7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016" gracePeriod=30 Jul 06 23:50:57.348863 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.348821 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:50:57.434039 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.434001 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") pod \"0031876a-4b74-4311-917d-8dba25b6df06\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " Jul 06 23:50:57.434223 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.434051 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-bound-sa-token\") pod \"0031876a-4b74-4311-917d-8dba25b6df06\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " Jul 06 23:50:57.434223 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.434080 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-installation-pull-secrets\") pod \"0031876a-4b74-4311-917d-8dba25b6df06\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " Jul 06 23:50:57.434223 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.434115 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0031876a-4b74-4311-917d-8dba25b6df06-ca-trust-extracted\") pod \"0031876a-4b74-4311-917d-8dba25b6df06\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " Jul 06 23:50:57.434223 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.434143 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-trusted-ca\") pod \"0031876a-4b74-4311-917d-8dba25b6df06\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " Jul 06 23:50:57.434223 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.434177 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-registry-certificates\") pod \"0031876a-4b74-4311-917d-8dba25b6df06\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " Jul 06 23:50:57.434223 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.434201 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6qls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-kube-api-access-l6qls\") pod \"0031876a-4b74-4311-917d-8dba25b6df06\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " Jul 06 23:50:57.434523 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.434467 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-image-registry-private-configuration\") pod \"0031876a-4b74-4311-917d-8dba25b6df06\" (UID: \"0031876a-4b74-4311-917d-8dba25b6df06\") " Jul 06 23:50:57.434728 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.434689 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0031876a-4b74-4311-917d-8dba25b6df06" (UID: "0031876a-4b74-4311-917d-8dba25b6df06"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:57.434728 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.434697 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0031876a-4b74-4311-917d-8dba25b6df06" (UID: "0031876a-4b74-4311-917d-8dba25b6df06"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:57.436891 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.436835 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0031876a-4b74-4311-917d-8dba25b6df06" (UID: "0031876a-4b74-4311-917d-8dba25b6df06"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:50:57.437039 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.437015 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0031876a-4b74-4311-917d-8dba25b6df06" (UID: "0031876a-4b74-4311-917d-8dba25b6df06"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:50:57.437318 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.437294 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0031876a-4b74-4311-917d-8dba25b6df06" (UID: "0031876a-4b74-4311-917d-8dba25b6df06"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:50:57.437381 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.437323 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-kube-api-access-l6qls" (OuterVolumeSpecName: "kube-api-access-l6qls") pod "0031876a-4b74-4311-917d-8dba25b6df06" (UID: "0031876a-4b74-4311-917d-8dba25b6df06"). InnerVolumeSpecName "kube-api-access-l6qls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:50:57.437417 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.437377 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0031876a-4b74-4311-917d-8dba25b6df06" (UID: "0031876a-4b74-4311-917d-8dba25b6df06"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:50:57.443159 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.443137 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0031876a-4b74-4311-917d-8dba25b6df06-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0031876a-4b74-4311-917d-8dba25b6df06" (UID: "0031876a-4b74-4311-917d-8dba25b6df06"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:50:57.535794 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.535754 2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-registry-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:50:57.535794 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.535788 2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-bound-sa-token\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:50:57.535794 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.535798 2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-installation-pull-secrets\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:50:57.536043 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.535809 2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0031876a-4b74-4311-917d-8dba25b6df06-ca-trust-extracted\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:50:57.536043 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.535817 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-trusted-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:50:57.536043 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.535825 2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0031876a-4b74-4311-917d-8dba25b6df06-registry-certificates\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:50:57.536043 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.535833 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6qls\" (UniqueName: \"kubernetes.io/projected/0031876a-4b74-4311-917d-8dba25b6df06-kube-api-access-l6qls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:50:57.536043 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.535843 2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0031876a-4b74-4311-917d-8dba25b6df06-image-registry-private-configuration\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:50:57.749977 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.749934 2558 generic.go:356] "Generic (PLEG): container finished" podID="0031876a-4b74-4311-917d-8dba25b6df06" containerID="7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016" exitCode=0 Jul 06 23:50:57.750146 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.749985 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" Jul 06 23:50:57.750146 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.750010 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" event={"ID":"0031876a-4b74-4311-917d-8dba25b6df06","Type":"ContainerDied","Data":"7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016"} Jul 06 23:50:57.750146 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.750094 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649bbb5647-76qxk" event={"ID":"0031876a-4b74-4311-917d-8dba25b6df06","Type":"ContainerDied","Data":"a95feeac9183dca180cfc9116a78fb7df06c4519076801d7cca7a2d8f08c9d72"} Jul 06 23:50:57.750146 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.750123 2558 scope.go:117] "RemoveContainer" containerID="7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016" Jul 06 23:50:57.766463 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.766443 2558 scope.go:117] "RemoveContainer" containerID="7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016" Jul 06 23:50:57.766781 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:50:57.766756 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016\": container with ID starting with 7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016 not found: ID does not exist" containerID="7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016" Jul 06 23:50:57.766868 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.766790 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016"} err="failed to get container status \"7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016\": rpc error: code = NotFound desc = could not find container \"7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016\": container with ID starting with 7e0e8eecca74f8199fe54b45d9aef9c85bee1e5489657d9c0ad2692f227d6016 not found: ID does not exist" Jul 06 23:50:57.770511 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.770486 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-649bbb5647-76qxk"] Jul 06 23:50:57.775780 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:57.775755 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-649bbb5647-76qxk"] Jul 06 23:50:58.049821 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:50:58.049742 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0031876a-4b74-4311-917d-8dba25b6df06" path="/var/lib/kubelet/pods/0031876a-4b74-4311-917d-8dba25b6df06/volumes" Jul 06 23:51:05.642394 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:05.642346 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" podUID="c988e8d2-033f-476d-a7fc-6e7d27acdef8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 23:51:08.746727 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:08.746692 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:10.430364 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:10.430325 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:10.531493 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:10.531451 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:15.642188 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:15.642143 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" podUID="c988e8d2-033f-476d-a7fc-6e7d27acdef8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 23:51:15.642584 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:15.642226 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" Jul 06 23:51:15.642667 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:15.642652 2558 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"045743c97cfd20711b3d48a335ae1009003a8d7d6645286633a12b8a5965eaba"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 06 23:51:15.642701 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:15.642687 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" podUID="c988e8d2-033f-476d-a7fc-6e7d27acdef8" containerName="service-proxy" containerID="cri-o://045743c97cfd20711b3d48a335ae1009003a8d7d6645286633a12b8a5965eaba" gracePeriod=30 Jul 06 23:51:15.809911 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:15.809881 2558 generic.go:356] "Generic (PLEG): container finished" podID="c988e8d2-033f-476d-a7fc-6e7d27acdef8" containerID="045743c97cfd20711b3d48a335ae1009003a8d7d6645286633a12b8a5965eaba" exitCode=2 Jul 06 23:51:15.810025 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:15.809957 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" event={"ID":"c988e8d2-033f-476d-a7fc-6e7d27acdef8","Type":"ContainerDied","Data":"045743c97cfd20711b3d48a335ae1009003a8d7d6645286633a12b8a5965eaba"} Jul 06 23:51:15.810025 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:15.810018 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f45c469-j7ln8" event={"ID":"c988e8d2-033f-476d-a7fc-6e7d27acdef8","Type":"ContainerStarted","Data":"e2f40dd85c334eaf1386d709dc7e9b1b776a61ec98f4e135972afb449f779f0c"} Jul 06 23:51:32.661611 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:32.661521 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:49.762268 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:49.762234 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:49.860241 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:49.860206 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:58.752107 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:58.752062 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:58.852136 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:58.852096 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:58.914436 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:58.914397 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:59.674590 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:59.674546 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:59.776759 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:51:59.776722 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:01.666375 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:01.666340 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:01.724966 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:01.724930 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:09.891687 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:09.891639 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:09.992149 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:09.992109 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:10.053392 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:10.053354 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:10.426644 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:10.426609 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:10.527185 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:10.527146 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:10.627093 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:10.627057 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:36.384806 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:36.384753 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:36.445518 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:36.445487 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:38.510797 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:38.510758 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:38.633725 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:52:38.633682 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:06.026759 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:06.026666 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:06.175808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:06.175769 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:10.205552 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:10.205519 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:10.303389 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:10.303356 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:11.339611 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:11.339577 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:11.494353 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:11.494313 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:11.604183 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:11.604095 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:13.981874 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:13.981807 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:14.397454 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:14.397366 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:16.382657 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:16.382608 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:16.483342 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:16.483309 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:20.786821 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:20.786778 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:20.884627 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:20.884588 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:21.211309 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.211280 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:21.325285 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.325251 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r5jvr"] Jul 06 23:53:21.325594 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.325577 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0031876a-4b74-4311-917d-8dba25b6df06" containerName="registry" Jul 06 23:53:21.325675 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.325598 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0031876a-4b74-4311-917d-8dba25b6df06" containerName="registry" Jul 06 23:53:21.325675 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.325659 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0031876a-4b74-4311-917d-8dba25b6df06" containerName="registry" Jul 06 23:53:21.328416 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.328394 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-r5jvr" Jul 06 23:53:21.331191 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.331169 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 23:53:21.331309 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.331261 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wscvn\"" Jul 06 23:53:21.331363 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.331352 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 23:53:21.339210 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.339183 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r5jvr"] Jul 06 23:53:21.431266 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.431230 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8zbw\" (UniqueName: \"kubernetes.io/projected/1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b-kube-api-access-g8zbw\") pod \"cert-manager-55b4d954d5-r5jvr\" (UID: \"1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b\") " pod="cert-manager/cert-manager-55b4d954d5-r5jvr" Jul 06 23:53:21.431266 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.431269 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b-bound-sa-token\") pod \"cert-manager-55b4d954d5-r5jvr\" (UID: \"1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b\") " pod="cert-manager/cert-manager-55b4d954d5-r5jvr" Jul 06 23:53:21.532428 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.532328 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8zbw\" (UniqueName: \"kubernetes.io/projected/1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b-kube-api-access-g8zbw\") pod \"cert-manager-55b4d954d5-r5jvr\" (UID: \"1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b\") " pod="cert-manager/cert-manager-55b4d954d5-r5jvr" Jul 06 23:53:21.532428 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.532364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b-bound-sa-token\") pod \"cert-manager-55b4d954d5-r5jvr\" (UID: \"1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b\") " pod="cert-manager/cert-manager-55b4d954d5-r5jvr" Jul 06 23:53:21.540760 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.540734 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b-bound-sa-token\") pod \"cert-manager-55b4d954d5-r5jvr\" (UID: \"1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b\") " pod="cert-manager/cert-manager-55b4d954d5-r5jvr" Jul 06 23:53:21.540941 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.540923 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8zbw\" (UniqueName: \"kubernetes.io/projected/1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b-kube-api-access-g8zbw\") pod \"cert-manager-55b4d954d5-r5jvr\" (UID: \"1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b\") " pod="cert-manager/cert-manager-55b4d954d5-r5jvr" Jul 06 23:53:21.637124 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.637084 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-r5jvr" Jul 06 23:53:21.768079 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:21.767988 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r5jvr"] Jul 06 23:53:21.770602 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:53:21.770572 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1bd2ab9a_2541_4cb3_b4a5_b82b35c28c7b.slice/crio-a6f239a1daa4544db46e7c80d81717b3b65297f5c05b233dd0ffd67ac1f4e8fc WatchSource:0}: Error finding container a6f239a1daa4544db46e7c80d81717b3b65297f5c05b233dd0ffd67ac1f4e8fc: Status 404 returned error can't find the container with id a6f239a1daa4544db46e7c80d81717b3b65297f5c05b233dd0ffd67ac1f4e8fc Jul 06 23:53:22.220523 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:22.220487 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-r5jvr" event={"ID":"1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b","Type":"ContainerStarted","Data":"a6f239a1daa4544db46e7c80d81717b3b65297f5c05b233dd0ffd67ac1f4e8fc"} Jul 06 23:53:23.673628 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.673596 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm"] Jul 06 23:53:23.682493 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.682380 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" Jul 06 23:53:23.686324 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.686247 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-kjgr8\"" Jul 06 23:53:23.686324 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.686390 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:53:23.686324 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.686390 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 23:53:23.686788 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.686543 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm"] Jul 06 23:53:23.753486 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.753452 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvhtf\" (UniqueName: \"kubernetes.io/projected/dd49fb7b-5ec3-442e-800f-0d94e7ee53d4-kube-api-access-xvhtf\") pod \"openshift-lws-operator-fd8ccff4c-cqmpm\" (UID: \"dd49fb7b-5ec3-442e-800f-0d94e7ee53d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" Jul 06 23:53:23.753651 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.753513 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd49fb7b-5ec3-442e-800f-0d94e7ee53d4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cqmpm\" (UID: \"dd49fb7b-5ec3-442e-800f-0d94e7ee53d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" Jul 06 23:53:23.854360 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.854315 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvhtf\" (UniqueName: \"kubernetes.io/projected/dd49fb7b-5ec3-442e-800f-0d94e7ee53d4-kube-api-access-xvhtf\") pod \"openshift-lws-operator-fd8ccff4c-cqmpm\" (UID: \"dd49fb7b-5ec3-442e-800f-0d94e7ee53d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" Jul 06 23:53:23.854544 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.854379 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd49fb7b-5ec3-442e-800f-0d94e7ee53d4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cqmpm\" (UID: \"dd49fb7b-5ec3-442e-800f-0d94e7ee53d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" Jul 06 23:53:23.854787 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.854714 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd49fb7b-5ec3-442e-800f-0d94e7ee53d4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cqmpm\" (UID: \"dd49fb7b-5ec3-442e-800f-0d94e7ee53d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" Jul 06 23:53:23.862954 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.862922 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvhtf\" (UniqueName: \"kubernetes.io/projected/dd49fb7b-5ec3-442e-800f-0d94e7ee53d4-kube-api-access-xvhtf\") pod \"openshift-lws-operator-fd8ccff4c-cqmpm\" (UID: \"dd49fb7b-5ec3-442e-800f-0d94e7ee53d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" Jul 06 23:53:23.875003 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.874965 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:23.975464 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.975341 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:23.995878 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:23.995809 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" Jul 06 23:53:25.101964 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:25.101936 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm"] Jul 06 23:53:25.105248 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:53:25.105219 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd49fb7b_5ec3_442e_800f_0d94e7ee53d4.slice/crio-b5d251f9b80e3aba62ec250465a82e555a5f49b00813380578fa818d513e75eb WatchSource:0}: Error finding container b5d251f9b80e3aba62ec250465a82e555a5f49b00813380578fa818d513e75eb: Status 404 returned error can't find the container with id b5d251f9b80e3aba62ec250465a82e555a5f49b00813380578fa818d513e75eb Jul 06 23:53:25.235311 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:25.235207 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-r5jvr" event={"ID":"1bd2ab9a-2541-4cb3-b4a5-b82b35c28c7b","Type":"ContainerStarted","Data":"4354a94b186a21daed719eba068dc5c42e30cbc2561e9d36980a78eb98b353d0"} Jul 06 23:53:25.236421 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:25.236398 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" event={"ID":"dd49fb7b-5ec3-442e-800f-0d94e7ee53d4","Type":"ContainerStarted","Data":"b5d251f9b80e3aba62ec250465a82e555a5f49b00813380578fa818d513e75eb"} Jul 06 23:53:25.252175 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:25.252116 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-r5jvr" podStartSLOduration=0.982434924 podStartE2EDuration="4.252099548s" totalImagesPullingTime="3.269664624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:25.251085951 +0000 UTC m=+285.761246909" watchObservedRunningTime="2026-07-06 23:53:25.252099548 +0000 UTC m=+285.762260504" Jul 06 23:53:27.207436 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:27.207375 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:28.250867 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:28.250805 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" event={"ID":"dd49fb7b-5ec3-442e-800f-0d94e7ee53d4","Type":"ContainerStarted","Data":"e12fa0de48642b3508eff27cac0cab4d90a58d1b714813364f072514b575847f"} Jul 06 23:53:28.268062 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:28.268011 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqmpm" podStartSLOduration=2.718037535 podStartE2EDuration="5.267991725s" totalImagesPullingTime="2.54995419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:53:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:28.266539647 +0000 UTC m=+288.776700620" watchObservedRunningTime="2026-07-06 23:53:28.267991725 +0000 UTC m=+288.778152681" Jul 06 23:53:28.945955 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:28.945918 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:29.047204 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:29.047170 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:30.711016 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:30.710972 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:39.944160 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:39.944124 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 06 23:53:39.951983 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:39.951954 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 23:53:39.952618 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:39.952597 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 06 23:53:43.474326 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:43.474283 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:47.404973 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:47.404933 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:47.506019 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:47.505986 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:48.045986 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.045942 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:48.209859 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.209804 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:48.310664 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.310571 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:48.364539 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.364507 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68"] Jul 06 23:53:48.366620 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.366603 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.371126 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.371101 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 23:53:48.371343 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.371145 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 23:53:48.371343 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.371222 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 23:53:48.371530 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.371405 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 23:53:48.371530 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.371461 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8sfwg\"" Jul 06 23:53:48.396599 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.396562 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68"] Jul 06 23:53:48.447053 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.447015 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.447434 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.447075 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.447434 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.447154 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-manifests\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.447434 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.447184 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pqcj\" (UniqueName: \"kubernetes.io/projected/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-kube-api-access-4pqcj\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.447434 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.447227 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-tmp\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.547984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.547944 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-manifests\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.547984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.547988 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pqcj\" (UniqueName: \"kubernetes.io/projected/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-kube-api-access-4pqcj\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.548256 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.548027 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-tmp\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.548256 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.548073 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.548256 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.548105 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.548444 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.548417 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-manifests\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.550493 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.550470 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-tmp\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.550717 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.550698 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.550837 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.550815 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.564537 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.564464 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pqcj\" (UniqueName: \"kubernetes.io/projected/71c36dbf-02fd-4924-a7ff-fd7d4490f2e2-kube-api-access-4pqcj\") pod \"opendatahub-operator-controller-manager-64648fd55c-rcl68\" (UID: \"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2\") " pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.585175 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.585128 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:48.671248 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.671213 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:48.677041 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.677022 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:48.777317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.777230 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:48.824016 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.823922 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68"] Jul 06 23:53:48.828566 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:53:48.828535 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c36dbf_02fd_4924_a7ff_fd7d4490f2e2.slice/crio-6dd2d50475bcec10b2d70aeb3c860afab8b56d5ce844eae4807551984aa48f6a WatchSource:0}: Error finding container 6dd2d50475bcec10b2d70aeb3c860afab8b56d5ce844eae4807551984aa48f6a: Status 404 returned error can't find the container with id 6dd2d50475bcec10b2d70aeb3c860afab8b56d5ce844eae4807551984aa48f6a Jul 06 23:53:48.830290 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.830275 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:53:48.872093 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.872061 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:48.975834 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:48.975800 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:49.325514 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:49.325478 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" event={"ID":"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2","Type":"ContainerStarted","Data":"6dd2d50475bcec10b2d70aeb3c860afab8b56d5ce844eae4807551984aa48f6a"} Jul 06 23:53:52.338801 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:52.338756 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" event={"ID":"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2","Type":"ContainerStarted","Data":"5f192b3e8a578a2f0e2764b8d86b3116f2ac7df1704e07bdd2cd4a784a256c71"} Jul 06 23:53:52.957642 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:52.957597 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:53.064527 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:53.064435 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:53:53.344375 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:53.344287 2558 generic.go:356] "Generic (PLEG): container finished" podID="71c36dbf-02fd-4924-a7ff-fd7d4490f2e2" containerID="5f192b3e8a578a2f0e2764b8d86b3116f2ac7df1704e07bdd2cd4a784a256c71" exitCode=0 Jul 06 23:53:53.344375 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:53.344346 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" event={"ID":"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2","Type":"ContainerDied","Data":"5f192b3e8a578a2f0e2764b8d86b3116f2ac7df1704e07bdd2cd4a784a256c71"} Jul 06 23:53:54.349678 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:54.349636 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" event={"ID":"71c36dbf-02fd-4924-a7ff-fd7d4490f2e2","Type":"ContainerStarted","Data":"81a01983156bfc159f8b379d141f8d805f9ed1bbc24dda40dd1b5af2b365411d"} Jul 06 23:53:54.350163 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:54.349716 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:53:54.372067 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:54.372008 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" podStartSLOduration=2.162149349 podStartE2EDuration="6.37199138s" totalImagesPullingTime="3.110209149s" totalInitContainerRuntime="1.099632882s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:53:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:54.371704139 +0000 UTC m=+314.881865110" watchObservedRunningTime="2026-07-06 23:53:54.37199138 +0000 UTC m=+314.882152405" Jul 06 23:53:59.546026 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:53:59.545978 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:00.624751 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:00.624716 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:05.355978 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:05.355942 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64648fd55c-rcl68" Jul 06 23:54:07.021314 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.021268 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:07.186239 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.186196 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:07.481284 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.481241 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:07.624508 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.624472 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x"] Jul 06 23:54:07.627659 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.627635 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:07.632097 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.632073 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5cbnd\"" Jul 06 23:54:07.632223 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.632106 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 23:54:07.632498 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.632481 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 23:54:07.632596 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.632501 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 23:54:07.632596 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.632538 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 23:54:07.644248 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.644219 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x"] Jul 06 23:54:07.674196 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.674161 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:07.787204 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.787108 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f554f49-1343-445a-a60a-49781ba73cac-tls-certs\") pod \"kube-auth-proxy-5c55d5ff9-scg7x\" (UID: \"9f554f49-1343-445a-a60a-49781ba73cac\") " pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:07.787204 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.787146 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f554f49-1343-445a-a60a-49781ba73cac-tmp\") pod \"kube-auth-proxy-5c55d5ff9-scg7x\" (UID: \"9f554f49-1343-445a-a60a-49781ba73cac\") " pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:07.787204 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.787196 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk949\" (UniqueName: \"kubernetes.io/projected/9f554f49-1343-445a-a60a-49781ba73cac-kube-api-access-rk949\") pod \"kube-auth-proxy-5c55d5ff9-scg7x\" (UID: \"9f554f49-1343-445a-a60a-49781ba73cac\") " pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:07.888661 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.888625 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f554f49-1343-445a-a60a-49781ba73cac-tls-certs\") pod \"kube-auth-proxy-5c55d5ff9-scg7x\" (UID: \"9f554f49-1343-445a-a60a-49781ba73cac\") " pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:07.888661 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.888662 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f554f49-1343-445a-a60a-49781ba73cac-tmp\") pod \"kube-auth-proxy-5c55d5ff9-scg7x\" (UID: \"9f554f49-1343-445a-a60a-49781ba73cac\") " pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:07.888961 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.888704 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk949\" (UniqueName: \"kubernetes.io/projected/9f554f49-1343-445a-a60a-49781ba73cac-kube-api-access-rk949\") pod \"kube-auth-proxy-5c55d5ff9-scg7x\" (UID: \"9f554f49-1343-445a-a60a-49781ba73cac\") " pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:07.891317 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.891291 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f554f49-1343-445a-a60a-49781ba73cac-tmp\") pod \"kube-auth-proxy-5c55d5ff9-scg7x\" (UID: \"9f554f49-1343-445a-a60a-49781ba73cac\") " pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:07.891422 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.891323 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f554f49-1343-445a-a60a-49781ba73cac-tls-certs\") pod \"kube-auth-proxy-5c55d5ff9-scg7x\" (UID: \"9f554f49-1343-445a-a60a-49781ba73cac\") " pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:07.899062 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.899038 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk949\" (UniqueName: \"kubernetes.io/projected/9f554f49-1343-445a-a60a-49781ba73cac-kube-api-access-rk949\") pod \"kube-auth-proxy-5c55d5ff9-scg7x\" (UID: \"9f554f49-1343-445a-a60a-49781ba73cac\") " pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:07.937215 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:07.937172 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" Jul 06 23:54:08.108483 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:08.108396 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x"] Jul 06 23:54:08.112417 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:54:08.112388 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f554f49_1343_445a_a60a_49781ba73cac.slice/crio-8bdd6c01319202bec26988e57c866a6c88c94fe4c4b2c161852f13947d45a7d5 WatchSource:0}: Error finding container 8bdd6c01319202bec26988e57c866a6c88c94fe4c4b2c161852f13947d45a7d5: Status 404 returned error can't find the container with id 8bdd6c01319202bec26988e57c866a6c88c94fe4c4b2c161852f13947d45a7d5 Jul 06 23:54:08.114099 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:08.113990 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:08.227812 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:08.227771 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:08.329503 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:08.329472 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:08.399326 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:08.399230 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" event={"ID":"9f554f49-1343-445a-a60a-49781ba73cac","Type":"ContainerStarted","Data":"8bdd6c01319202bec26988e57c866a6c88c94fe4c4b2c161852f13947d45a7d5"} Jul 06 23:54:08.471634 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:08.471592 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:08.910067 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:08.910020 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:09.216481 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:09.216440 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:09.314730 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:09.314691 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:09.596617 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:09.596524 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:10.197189 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:10.197153 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:11.411693 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:11.411652 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" event={"ID":"9f554f49-1343-445a-a60a-49781ba73cac","Type":"ContainerStarted","Data":"0783c8104d656cb0cb8b6e1a42fe22adc575946408d76fa1962a43d4ac575339"} Jul 06 23:54:14.409059 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:14.409028 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:14.510662 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:14.510623 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:14.814005 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:14.813972 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:14.946864 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:14.946805 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:15.062435 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.062378 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c55d5ff9-scg7x" podStartSLOduration=5.764228984 podStartE2EDuration="8.062360874s" totalImagesPullingTime="2.29813189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:11.4291992 +0000 UTC m=+331.939360159" watchObservedRunningTime="2026-07-06 23:54:15.062360874 +0000 UTC m=+335.572521863" Jul 06 23:54:15.062597 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.062544 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n"] Jul 06 23:54:15.065377 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.065317 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.068356 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.068326 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 23:54:15.068454 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.068414 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 23:54:15.068779 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.068766 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fd9n7\"" Jul 06 23:54:15.069024 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.069007 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 23:54:15.069108 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.069091 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 23:54:15.080801 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.080777 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n"] Jul 06 23:54:15.249268 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.249228 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.249449 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.249275 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.249449 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.249300 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w44j2\" (UniqueName: \"kubernetes.io/projected/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-kube-api-access-w44j2\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.249449 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.249378 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.249449 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.249441 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.249621 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.249504 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.249621 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.249550 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.313330 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.313294 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:15.350399 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.350303 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.350399 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.350358 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.350399 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.350390 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w44j2\" (UniqueName: \"kubernetes.io/projected/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-kube-api-access-w44j2\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.350679 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.350416 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.350679 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.350465 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.350679 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.350498 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.350679 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.350526 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.351370 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.351332 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.353163 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.353137 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.353275 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.353148 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.353316 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.353294 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.353498 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.353474 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.362251 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.362221 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.362396 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.362379 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w44j2\" (UniqueName: \"kubernetes.io/projected/f4542275-7cb0-4a4b-8f52-6f9acc49c7bb-kube-api-access-w44j2\") pod \"istiod-openshift-gateway-94bb8fbfd-gdl6n\" (UID: \"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.374434 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.374409 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:15.514566 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:15.514537 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n"] Jul 06 23:54:15.516568 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:54:15.516539 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4542275_7cb0_4a4b_8f52_6f9acc49c7bb.slice/crio-f3bb4dba9d65091302b1eb314e48ec2866590362777a14149365fab324e9d2b5 WatchSource:0}: Error finding container f3bb4dba9d65091302b1eb314e48ec2866590362777a14149365fab324e9d2b5: Status 404 returned error can't find the container with id f3bb4dba9d65091302b1eb314e48ec2866590362777a14149365fab324e9d2b5 Jul 06 23:54:16.434778 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:16.434734 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" event={"ID":"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb","Type":"ContainerStarted","Data":"f3bb4dba9d65091302b1eb314e48ec2866590362777a14149365fab324e9d2b5"} Jul 06 23:54:17.592309 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:17.592267 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 23:54:17.592689 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:17.592357 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 23:54:17.786093 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:17.786050 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:17.877030 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:17.876931 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:18.292172 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:18.292138 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:18.413510 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:18.413470 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:18.444706 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:18.444671 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" event={"ID":"f4542275-7cb0-4a4b-8f52-6f9acc49c7bb","Type":"ContainerStarted","Data":"de8a5ac60849b2a599000c1bb540594ac52bfd67875a4a23525a1987acd5a5b8"} Jul 06 23:54:18.444908 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:18.444883 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:18.446473 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:18.446438 2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-gdl6n container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 23:54:18.446572 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:18.446494 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" podUID="f4542275-7cb0-4a4b-8f52-6f9acc49c7bb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 23:54:18.491329 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:18.491273 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" podStartSLOduration=1.417765792 podStartE2EDuration="3.491252152s" totalImagesPullingTime="2.07348636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:18.490676844 +0000 UTC m=+339.000837826" watchObservedRunningTime="2026-07-06 23:54:18.491252152 +0000 UTC m=+339.001413109" Jul 06 23:54:19.122402 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:19.122372 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:19.204608 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:19.204569 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:19.449516 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:19.449488 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdl6n" Jul 06 23:54:48.400166 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:48.400125 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:48.580130 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:48.580085 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:54.812629 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:54.812583 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:54.914341 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:54:54.914301 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:08.499637 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.499583 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:08.589978 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.589942 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:08.621613 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.621580 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5qvxn"] Jul 06 23:55:08.624749 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.624724 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" Jul 06 23:55:08.627983 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.627954 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 23:55:08.629175 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.629155 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-p7fnw\"" Jul 06 23:55:08.629292 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.629180 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 23:55:08.635441 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.635368 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5qvxn"] Jul 06 23:55:08.652930 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.652895 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htfng\" (UniqueName: \"kubernetes.io/projected/71f58758-8520-4575-b377-c6a28832adb5-kube-api-access-htfng\") pod \"kuadrant-operator-catalog-5qvxn\" (UID: \"71f58758-8520-4575-b377-c6a28832adb5\") " pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" Jul 06 23:55:08.680099 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.680061 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:08.754110 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.754029 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htfng\" (UniqueName: \"kubernetes.io/projected/71f58758-8520-4575-b377-c6a28832adb5-kube-api-access-htfng\") pod \"kuadrant-operator-catalog-5qvxn\" (UID: \"71f58758-8520-4575-b377-c6a28832adb5\") " pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" Jul 06 23:55:08.762698 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.762665 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htfng\" (UniqueName: \"kubernetes.io/projected/71f58758-8520-4575-b377-c6a28832adb5-kube-api-access-htfng\") pod \"kuadrant-operator-catalog-5qvxn\" (UID: \"71f58758-8520-4575-b377-c6a28832adb5\") " pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" Jul 06 23:55:08.935818 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.935773 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" Jul 06 23:55:08.979351 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:08.979313 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5qvxn"] Jul 06 23:55:09.071705 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:09.071676 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5qvxn"] Jul 06 23:55:09.075335 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:55:09.075299 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f58758_8520_4575_b377_c6a28832adb5.slice/crio-bfc3b9935985b25908b5894f5a25b9e4feb67fc1f1319abb69bb393c4a16c44a WatchSource:0}: Error finding container bfc3b9935985b25908b5894f5a25b9e4feb67fc1f1319abb69bb393c4a16c44a: Status 404 returned error can't find the container with id bfc3b9935985b25908b5894f5a25b9e4feb67fc1f1319abb69bb393c4a16c44a Jul 06 23:55:09.420006 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:09.419915 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:09.621928 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:09.621896 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:09.628969 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:09.628880 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" event={"ID":"71f58758-8520-4575-b377-c6a28832adb5","Type":"ContainerStarted","Data":"bfc3b9935985b25908b5894f5a25b9e4feb67fc1f1319abb69bb393c4a16c44a"} Jul 06 23:55:11.637883 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:11.637751 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" event={"ID":"71f58758-8520-4575-b377-c6a28832adb5","Type":"ContainerStarted","Data":"f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c"} Jul 06 23:55:11.638306 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:11.637892 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" podUID="71f58758-8520-4575-b377-c6a28832adb5" containerName="registry-server" containerID="cri-o://f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c" gracePeriod=2 Jul 06 23:55:11.653430 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:11.653381 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" podStartSLOduration=1.475359377 podStartE2EDuration="3.653364842s" totalImagesPullingTime="2.178005465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:11.652826655 +0000 UTC m=+392.162987615" watchObservedRunningTime="2026-07-06 23:55:11.653364842 +0000 UTC m=+392.163525799" Jul 06 23:55:11.886383 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:11.886351 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" Jul 06 23:55:11.979502 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:11.979465 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htfng\" (UniqueName: \"kubernetes.io/projected/71f58758-8520-4575-b377-c6a28832adb5-kube-api-access-htfng\") pod \"71f58758-8520-4575-b377-c6a28832adb5\" (UID: \"71f58758-8520-4575-b377-c6a28832adb5\") " Jul 06 23:55:11.981833 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:11.981805 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f58758-8520-4575-b377-c6a28832adb5-kube-api-access-htfng" (OuterVolumeSpecName: "kube-api-access-htfng") pod "71f58758-8520-4575-b377-c6a28832adb5" (UID: "71f58758-8520-4575-b377-c6a28832adb5"). InnerVolumeSpecName "kube-api-access-htfng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:55:12.080527 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:12.080499 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htfng\" (UniqueName: \"kubernetes.io/projected/71f58758-8520-4575-b377-c6a28832adb5-kube-api-access-htfng\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:55:12.642424 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:12.642387 2558 generic.go:356] "Generic (PLEG): container finished" podID="71f58758-8520-4575-b377-c6a28832adb5" containerID="f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c" exitCode=0 Jul 06 23:55:12.642879 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:12.642432 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" Jul 06 23:55:12.642879 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:12.642443 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" event={"ID":"71f58758-8520-4575-b377-c6a28832adb5","Type":"ContainerDied","Data":"f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c"} Jul 06 23:55:12.642879 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:12.642484 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5qvxn" event={"ID":"71f58758-8520-4575-b377-c6a28832adb5","Type":"ContainerDied","Data":"bfc3b9935985b25908b5894f5a25b9e4feb67fc1f1319abb69bb393c4a16c44a"} Jul 06 23:55:12.642879 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:12.642505 2558 scope.go:117] "RemoveContainer" containerID="f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c" Jul 06 23:55:12.652765 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:12.652744 2558 scope.go:117] "RemoveContainer" containerID="f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c" Jul 06 23:55:12.653111 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:55:12.653089 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c\": container with ID starting with f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c not found: ID does not exist" containerID="f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c" Jul 06 23:55:12.653168 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:12.653121 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c"} err="failed to get container status \"f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c\": rpc error: code = NotFound desc = could not find container \"f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c\": container with ID starting with f0b832457c374d5a90f6087875252e40fccbb6ad93d736c10574f3539d0dd67c not found: ID does not exist" Jul 06 23:55:12.660704 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:12.660674 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5qvxn"] Jul 06 23:55:12.665045 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:12.665022 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5qvxn"] Jul 06 23:55:14.050339 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:14.050303 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71f58758-8520-4575-b377-c6a28832adb5" path="/var/lib/kubelet/pods/71f58758-8520-4575-b377-c6a28832adb5/volumes" Jul 06 23:55:29.270087 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:29.270046 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:31.677626 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.677586 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp"] Jul 06 23:55:31.678048 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.677868 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f58758-8520-4575-b377-c6a28832adb5" containerName="registry-server" Jul 06 23:55:31.678048 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.677880 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f58758-8520-4575-b377-c6a28832adb5" containerName="registry-server" Jul 06 23:55:31.678048 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.677936 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="71f58758-8520-4575-b377-c6a28832adb5" containerName="registry-server" Jul 06 23:55:31.680537 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.680519 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:31.684589 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.684568 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-62d8p\"" Jul 06 23:55:31.685713 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.685689 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 23:55:31.685886 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.685774 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 23:55:31.698988 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.698962 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp"] Jul 06 23:55:31.839894 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.839823 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxqkw\" (UniqueName: \"kubernetes.io/projected/2eb6f6a7-55d0-4db9-af08-96fa02419a60-kube-api-access-fxqkw\") pod \"limitador-operator-controller-manager-859d7b78bd-jwdvp\" (UID: \"2eb6f6a7-55d0-4db9-af08-96fa02419a60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:31.940648 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.940537 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxqkw\" (UniqueName: \"kubernetes.io/projected/2eb6f6a7-55d0-4db9-af08-96fa02419a60-kube-api-access-fxqkw\") pod \"limitador-operator-controller-manager-859d7b78bd-jwdvp\" (UID: \"2eb6f6a7-55d0-4db9-af08-96fa02419a60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:31.950838 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.950805 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxqkw\" (UniqueName: \"kubernetes.io/projected/2eb6f6a7-55d0-4db9-af08-96fa02419a60-kube-api-access-fxqkw\") pod \"limitador-operator-controller-manager-859d7b78bd-jwdvp\" (UID: \"2eb6f6a7-55d0-4db9-af08-96fa02419a60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:31.990841 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:31.990801 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:32.128723 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:32.128689 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp"] Jul 06 23:55:32.131729 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:55:32.131706 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eb6f6a7_55d0_4db9_af08_96fa02419a60.slice/crio-676093be0e275450e3af81cf24848b93e45dea9950eff253265f793f2c5d1187 WatchSource:0}: Error finding container 676093be0e275450e3af81cf24848b93e45dea9950eff253265f793f2c5d1187: Status 404 returned error can't find the container with id 676093be0e275450e3af81cf24848b93e45dea9950eff253265f793f2c5d1187 Jul 06 23:55:32.473015 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:32.472981 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:32.724433 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:32.724291 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" event={"ID":"2eb6f6a7-55d0-4db9-af08-96fa02419a60","Type":"ContainerStarted","Data":"676093be0e275450e3af81cf24848b93e45dea9950eff253265f793f2c5d1187"} Jul 06 23:55:33.266835 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:33.266791 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:33.730249 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:33.730214 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" event={"ID":"2eb6f6a7-55d0-4db9-af08-96fa02419a60","Type":"ContainerStarted","Data":"5d35af133bbacd95677335c891e2372c26837d5158cd9d9e6776ec56cc342959"} Jul 06 23:55:33.730666 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:33.730336 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:33.749791 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:33.749730 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" podStartSLOduration=1.495622771 podStartE2EDuration="2.749713704s" totalImagesPullingTime="1.254090933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:33.747608975 +0000 UTC m=+414.257769932" watchObservedRunningTime="2026-07-06 23:55:33.749713704 +0000 UTC m=+414.259874727" Jul 06 23:55:36.643560 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:36.643529 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:36.964320 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:36.964281 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:39.446388 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:39.446348 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:41.120768 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.120732 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:41.152770 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.152732 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8"] Jul 06 23:55:41.157090 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.157069 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:41.160152 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.160135 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gk8sk\"" Jul 06 23:55:41.176260 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.176224 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8"] Jul 06 23:55:41.318802 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.318749 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4bd1ab0-aa87-4077-9bf4-aa1668523342-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" (UID: \"f4bd1ab0-aa87-4077-9bf4-aa1668523342\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:41.318802 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.318804 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn9d4\" (UniqueName: \"kubernetes.io/projected/f4bd1ab0-aa87-4077-9bf4-aa1668523342-kube-api-access-cn9d4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" (UID: \"f4bd1ab0-aa87-4077-9bf4-aa1668523342\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:41.419584 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.419488 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4bd1ab0-aa87-4077-9bf4-aa1668523342-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" (UID: \"f4bd1ab0-aa87-4077-9bf4-aa1668523342\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:41.419584 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.419533 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn9d4\" (UniqueName: \"kubernetes.io/projected/f4bd1ab0-aa87-4077-9bf4-aa1668523342-kube-api-access-cn9d4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" (UID: \"f4bd1ab0-aa87-4077-9bf4-aa1668523342\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:41.419915 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.419896 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4bd1ab0-aa87-4077-9bf4-aa1668523342-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" (UID: \"f4bd1ab0-aa87-4077-9bf4-aa1668523342\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:41.437901 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.437844 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn9d4\" (UniqueName: \"kubernetes.io/projected/f4bd1ab0-aa87-4077-9bf4-aa1668523342-kube-api-access-cn9d4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" (UID: \"f4bd1ab0-aa87-4077-9bf4-aa1668523342\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:41.466876 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.466817 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:41.608831 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.608798 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8"] Jul 06 23:55:41.612555 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:55:41.612528 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4bd1ab0_aa87_4077_9bf4_aa1668523342.slice/crio-2002dcd4d093d53f36eeeeaa675e9d842aac4229d32f1ef7b12ed0061ac320a6 WatchSource:0}: Error finding container 2002dcd4d093d53f36eeeeaa675e9d842aac4229d32f1ef7b12ed0061ac320a6: Status 404 returned error can't find the container with id 2002dcd4d093d53f36eeeeaa675e9d842aac4229d32f1ef7b12ed0061ac320a6 Jul 06 23:55:41.765842 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:41.765806 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" event={"ID":"f4bd1ab0-aa87-4077-9bf4-aa1668523342","Type":"ContainerStarted","Data":"2002dcd4d093d53f36eeeeaa675e9d842aac4229d32f1ef7b12ed0061ac320a6"} Jul 06 23:55:42.056019 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:42.055932 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:44.289723 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:44.289657 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:44.736621 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:44.736589 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:45.864738 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:45.864680 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:45.968741 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:45.968698 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:46.788725 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:46.788685 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" event={"ID":"f4bd1ab0-aa87-4077-9bf4-aa1668523342","Type":"ContainerStarted","Data":"9f8299ff56af845b73c288f1df945fe5960a30caf8ab1b9b30148225d490cf24"} Jul 06 23:55:46.788945 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:46.788816 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:46.809867 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:46.809803 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" podStartSLOduration=1.4375334309999999 podStartE2EDuration="5.80978985s" totalImagesPullingTime="4.372256419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:46.808396876 +0000 UTC m=+427.318557832" watchObservedRunningTime="2026-07-06 23:55:46.80978985 +0000 UTC m=+427.319950833" Jul 06 23:55:48.839195 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:48.839149 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:49.668634 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:49.668593 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:49.785004 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:49.784955 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:49.904502 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:49.904453 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:49.986371 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:49.986320 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:50.277871 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:50.277762 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:50.422984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:50.422945 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:54.475958 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:54.475920 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:54.574076 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:54.574028 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:57.795445 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:57.795411 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:59.607301 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.607264 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8"] Jul 06 23:55:59.607721 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.607464 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" containerName="manager" containerID="cri-o://9f8299ff56af845b73c288f1df945fe5960a30caf8ab1b9b30148225d490cf24" gracePeriod=2 Jul 06 23:55:59.611700 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.611670 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8"] Jul 06 23:55:59.612708 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:55:59.612673 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" is forbidden: User \"system:node:ip-10-0-130-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-213.ec2.internal' and this object" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:59.634613 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.634583 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb"] Jul 06 23:55:59.634892 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.634868 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" containerName="manager" Jul 06 23:55:59.634892 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.634883 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" containerName="manager" Jul 06 23:55:59.634992 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.634947 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" containerName="manager" Jul 06 23:55:59.637747 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.637723 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:55:59.640104 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:55:59.640074 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" is forbidden: User \"system:node:ip-10-0-130-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-213.ec2.internal' and this object" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:59.642447 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.642427 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp"] Jul 06 23:55:59.642650 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.642628 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" podUID="2eb6f6a7-55d0-4db9-af08-96fa02419a60" containerName="manager" containerID="cri-o://5d35af133bbacd95677335c891e2372c26837d5158cd9d9e6776ec56cc342959" gracePeriod=2 Jul 06 23:55:59.651490 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.651464 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb"] Jul 06 23:55:59.656548 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.656517 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp"] Jul 06 23:55:59.662728 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.662704 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vfskb\" (UID: \"8592a4a2-a6be-4c7a-8a4f-8324d1c22569\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:55:59.662879 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.662789 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvlxh\" (UniqueName: \"kubernetes.io/projected/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-kube-api-access-tvlxh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vfskb\" (UID: \"8592a4a2-a6be-4c7a-8a4f-8324d1c22569\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:55:59.673675 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.673647 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2"] Jul 06 23:55:59.674225 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.674201 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eb6f6a7-55d0-4db9-af08-96fa02419a60" containerName="manager" Jul 06 23:55:59.674225 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.674225 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eb6f6a7-55d0-4db9-af08-96fa02419a60" containerName="manager" Jul 06 23:55:59.674398 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.674288 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eb6f6a7-55d0-4db9-af08-96fa02419a60" containerName="manager" Jul 06 23:55:59.677517 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.677499 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2" Jul 06 23:55:59.688463 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.688431 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2"] Jul 06 23:55:59.708975 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:55:59.708935 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jwdvp\" is forbidden: User \"system:node:ip-10-0-130-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-213.ec2.internal' and this object" podUID="2eb6f6a7-55d0-4db9-af08-96fa02419a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:59.711133 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:55:59.711090 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" is forbidden: User \"system:node:ip-10-0-130-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-213.ec2.internal' and this object" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:59.763922 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.763886 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62zr4\" (UniqueName: \"kubernetes.io/projected/ad21f8cc-7347-469b-9006-488d24ea3427-kube-api-access-62zr4\") pod \"limitador-operator-controller-manager-859d7b78bd-phpx2\" (UID: \"ad21f8cc-7347-469b-9006-488d24ea3427\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2" Jul 06 23:55:59.764041 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.763954 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vfskb\" (UID: \"8592a4a2-a6be-4c7a-8a4f-8324d1c22569\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:55:59.764041 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.764004 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvlxh\" (UniqueName: \"kubernetes.io/projected/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-kube-api-access-tvlxh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vfskb\" (UID: \"8592a4a2-a6be-4c7a-8a4f-8324d1c22569\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:55:59.764407 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.764383 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vfskb\" (UID: \"8592a4a2-a6be-4c7a-8a4f-8324d1c22569\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:55:59.776484 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.776417 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvlxh\" (UniqueName: \"kubernetes.io/projected/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-kube-api-access-tvlxh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vfskb\" (UID: \"8592a4a2-a6be-4c7a-8a4f-8324d1c22569\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:55:59.843489 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.843456 2558 generic.go:356] "Generic (PLEG): container finished" podID="2eb6f6a7-55d0-4db9-af08-96fa02419a60" containerID="5d35af133bbacd95677335c891e2372c26837d5158cd9d9e6776ec56cc342959" exitCode=0 Jul 06 23:55:59.845355 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.845329 2558 generic.go:356] "Generic (PLEG): container finished" podID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" containerID="9f8299ff56af845b73c288f1df945fe5960a30caf8ab1b9b30148225d490cf24" exitCode=0 Jul 06 23:55:59.865283 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.865253 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62zr4\" (UniqueName: \"kubernetes.io/projected/ad21f8cc-7347-469b-9006-488d24ea3427-kube-api-access-62zr4\") pod \"limitador-operator-controller-manager-859d7b78bd-phpx2\" (UID: \"ad21f8cc-7347-469b-9006-488d24ea3427\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2" Jul 06 23:55:59.878687 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.877319 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62zr4\" (UniqueName: \"kubernetes.io/projected/ad21f8cc-7347-469b-9006-488d24ea3427-kube-api-access-62zr4\") pod \"limitador-operator-controller-manager-859d7b78bd-phpx2\" (UID: \"ad21f8cc-7347-469b-9006-488d24ea3427\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2" Jul 06 23:55:59.878936 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.878914 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:59.881522 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:55:59.881493 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" is forbidden: User \"system:node:ip-10-0-130-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-213.ec2.internal' and this object" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:59.881782 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.881765 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:59.883663 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:55:59.883641 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jwdvp\" is forbidden: User \"system:node:ip-10-0-130-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-213.ec2.internal' and this object" podUID="2eb6f6a7-55d0-4db9-af08-96fa02419a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:59.885864 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:55:59.885826 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" is forbidden: User \"system:node:ip-10-0-130-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-213.ec2.internal' and this object" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:55:59.887980 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:55:59.887958 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jwdvp\" is forbidden: User \"system:node:ip-10-0-130-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-213.ec2.internal' and this object" podUID="2eb6f6a7-55d0-4db9-af08-96fa02419a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:55:59.965557 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.965525 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4bd1ab0-aa87-4077-9bf4-aa1668523342-extensions-socket-volume\") pod \"f4bd1ab0-aa87-4077-9bf4-aa1668523342\" (UID: \"f4bd1ab0-aa87-4077-9bf4-aa1668523342\") " Jul 06 23:55:59.965740 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.965565 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn9d4\" (UniqueName: \"kubernetes.io/projected/f4bd1ab0-aa87-4077-9bf4-aa1668523342-kube-api-access-cn9d4\") pod \"f4bd1ab0-aa87-4077-9bf4-aa1668523342\" (UID: \"f4bd1ab0-aa87-4077-9bf4-aa1668523342\") " Jul 06 23:55:59.965740 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.965628 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxqkw\" (UniqueName: \"kubernetes.io/projected/2eb6f6a7-55d0-4db9-af08-96fa02419a60-kube-api-access-fxqkw\") pod \"2eb6f6a7-55d0-4db9-af08-96fa02419a60\" (UID: \"2eb6f6a7-55d0-4db9-af08-96fa02419a60\") " Jul 06 23:55:59.966126 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.966102 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bd1ab0-aa87-4077-9bf4-aa1668523342-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f4bd1ab0-aa87-4077-9bf4-aa1668523342" (UID: "f4bd1ab0-aa87-4077-9bf4-aa1668523342"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:55:59.967931 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.967895 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4bd1ab0-aa87-4077-9bf4-aa1668523342-kube-api-access-cn9d4" (OuterVolumeSpecName: "kube-api-access-cn9d4") pod "f4bd1ab0-aa87-4077-9bf4-aa1668523342" (UID: "f4bd1ab0-aa87-4077-9bf4-aa1668523342"). InnerVolumeSpecName "kube-api-access-cn9d4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:55:59.967931 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:55:59.967903 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eb6f6a7-55d0-4db9-af08-96fa02419a60-kube-api-access-fxqkw" (OuterVolumeSpecName: "kube-api-access-fxqkw") pod "2eb6f6a7-55d0-4db9-af08-96fa02419a60" (UID: "2eb6f6a7-55d0-4db9-af08-96fa02419a60"). InnerVolumeSpecName "kube-api-access-fxqkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:56:00.030092 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.030053 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:56:00.035890 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.035864 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2" Jul 06 23:56:00.051438 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:56:00.051402 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7jbq8\" is forbidden: User \"system:node:ip-10-0-130-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-213.ec2.internal' and this object" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:56:00.052018 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.051999 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eb6f6a7-55d0-4db9-af08-96fa02419a60" path="/var/lib/kubelet/pods/2eb6f6a7-55d0-4db9-af08-96fa02419a60/volumes" Jul 06 23:56:00.052384 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.052371 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4bd1ab0-aa87-4077-9bf4-aa1668523342" path="/var/lib/kubelet/pods/f4bd1ab0-aa87-4077-9bf4-aa1668523342/volumes" Jul 06 23:56:00.054005 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:56:00.053980 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jwdvp\" is forbidden: User \"system:node:ip-10-0-130-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-213.ec2.internal' and this object" podUID="2eb6f6a7-55d0-4db9-af08-96fa02419a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:56:00.066756 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.066722 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4bd1ab0-aa87-4077-9bf4-aa1668523342-extensions-socket-volume\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:56:00.066756 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.066757 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn9d4\" (UniqueName: \"kubernetes.io/projected/f4bd1ab0-aa87-4077-9bf4-aa1668523342-kube-api-access-cn9d4\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:56:00.066993 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.066772 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxqkw\" (UniqueName: \"kubernetes.io/projected/2eb6f6a7-55d0-4db9-af08-96fa02419a60-kube-api-access-fxqkw\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:56:00.180934 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.180899 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb"] Jul 06 23:56:00.185348 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:56:00.185317 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8592a4a2_a6be_4c7a_8a4f_8324d1c22569.slice/crio-8c5dc2df3640d3bbca25be8006812074bdac94e4f4de0ce3fa07e41c7bafb54d WatchSource:0}: Error finding container 8c5dc2df3640d3bbca25be8006812074bdac94e4f4de0ce3fa07e41c7bafb54d: Status 404 returned error can't find the container with id 8c5dc2df3640d3bbca25be8006812074bdac94e4f4de0ce3fa07e41c7bafb54d Jul 06 23:56:00.201828 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.201796 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2"] Jul 06 23:56:00.205167 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:56:00.205133 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad21f8cc_7347_469b_9006_488d24ea3427.slice/crio-5fccc7b609c028fc751dfa00202a50d69eb5bb916c4f2857c70c2019eb4b4f00 WatchSource:0}: Error finding container 5fccc7b609c028fc751dfa00202a50d69eb5bb916c4f2857c70c2019eb4b4f00: Status 404 returned error can't find the container with id 5fccc7b609c028fc751dfa00202a50d69eb5bb916c4f2857c70c2019eb4b4f00 Jul 06 23:56:00.850981 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.850907 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7jbq8" Jul 06 23:56:00.850981 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.850923 2558 scope.go:117] "RemoveContainer" containerID="9f8299ff56af845b73c288f1df945fe5960a30caf8ab1b9b30148225d490cf24" Jul 06 23:56:00.852719 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.852690 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" event={"ID":"8592a4a2-a6be-4c7a-8a4f-8324d1c22569","Type":"ContainerStarted","Data":"b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845"} Jul 06 23:56:00.852836 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.852752 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:56:00.852836 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.852778 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" event={"ID":"8592a4a2-a6be-4c7a-8a4f-8324d1c22569","Type":"ContainerStarted","Data":"8c5dc2df3640d3bbca25be8006812074bdac94e4f4de0ce3fa07e41c7bafb54d"} Jul 06 23:56:00.854634 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.854601 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2" event={"ID":"ad21f8cc-7347-469b-9006-488d24ea3427","Type":"ContainerStarted","Data":"fb495d8724dce5bd1e3dc738d387792240578150b665d46e6345e70fcfcebbcd"} Jul 06 23:56:00.854732 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.854642 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2" event={"ID":"ad21f8cc-7347-469b-9006-488d24ea3427","Type":"ContainerStarted","Data":"5fccc7b609c028fc751dfa00202a50d69eb5bb916c4f2857c70c2019eb4b4f00"} Jul 06 23:56:00.854732 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.854664 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2" Jul 06 23:56:00.856045 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.856029 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jwdvp" Jul 06 23:56:00.860269 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.860247 2558 scope.go:117] "RemoveContainer" containerID="5d35af133bbacd95677335c891e2372c26837d5158cd9d9e6776ec56cc342959" Jul 06 23:56:00.882270 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.882183 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" podStartSLOduration=1.882166859 podStartE2EDuration="1.882166859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:00.882102768 +0000 UTC m=+441.392263743" watchObservedRunningTime="2026-07-06 23:56:00.882166859 +0000 UTC m=+441.392327797" Jul 06 23:56:00.920369 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.920313 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2" podStartSLOduration=1.920297626 podStartE2EDuration="1.920297626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:00.918996989 +0000 UTC m=+441.429157946" watchObservedRunningTime="2026-07-06 23:56:00.920297626 +0000 UTC m=+441.430458653" Jul 06 23:56:00.968370 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:00.968329 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:01.123297 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:01.123264 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:01.231413 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:01.231381 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:01.336390 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:01.336346 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:05.285119 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:05.285074 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:05.466195 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:05.466147 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:08.846752 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:08.846713 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:08.912960 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:08.912919 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:09.245989 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:09.245958 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:09.303649 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:09.303603 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:11.864452 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:11.864421 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:56:11.864878 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:11.864496 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phpx2" Jul 06 23:56:13.600000 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:13.599962 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:14.044592 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:14.044555 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:14.103471 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:14.103435 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:15.870682 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:15.870644 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb"] Jul 06 23:56:15.871171 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:15.870889 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" podUID="8592a4a2-a6be-4c7a-8a4f-8324d1c22569" containerName="manager" containerID="cri-o://b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845" gracePeriod=10 Jul 06 23:56:16.122080 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.122006 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:56:16.181247 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.181209 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvlxh\" (UniqueName: \"kubernetes.io/projected/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-kube-api-access-tvlxh\") pod \"8592a4a2-a6be-4c7a-8a4f-8324d1c22569\" (UID: \"8592a4a2-a6be-4c7a-8a4f-8324d1c22569\") " Jul 06 23:56:16.181430 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.181305 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-extensions-socket-volume\") pod \"8592a4a2-a6be-4c7a-8a4f-8324d1c22569\" (UID: \"8592a4a2-a6be-4c7a-8a4f-8324d1c22569\") " Jul 06 23:56:16.181653 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.181629 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8592a4a2-a6be-4c7a-8a4f-8324d1c22569" (UID: "8592a4a2-a6be-4c7a-8a4f-8324d1c22569"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:56:16.183469 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.183451 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-kube-api-access-tvlxh" (OuterVolumeSpecName: "kube-api-access-tvlxh") pod "8592a4a2-a6be-4c7a-8a4f-8324d1c22569" (UID: "8592a4a2-a6be-4c7a-8a4f-8324d1c22569"). InnerVolumeSpecName "kube-api-access-tvlxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:56:16.281986 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.281942 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvlxh\" (UniqueName: \"kubernetes.io/projected/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-kube-api-access-tvlxh\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:56:16.281986 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.281978 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8592a4a2-a6be-4c7a-8a4f-8324d1c22569-extensions-socket-volume\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:56:16.926739 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.926707 2558 generic.go:356] "Generic (PLEG): container finished" podID="8592a4a2-a6be-4c7a-8a4f-8324d1c22569" containerID="b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845" exitCode=0 Jul 06 23:56:16.927251 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.926763 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" Jul 06 23:56:16.927251 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.926765 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" event={"ID":"8592a4a2-a6be-4c7a-8a4f-8324d1c22569","Type":"ContainerDied","Data":"b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845"} Jul 06 23:56:16.927251 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.926910 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb" event={"ID":"8592a4a2-a6be-4c7a-8a4f-8324d1c22569","Type":"ContainerDied","Data":"8c5dc2df3640d3bbca25be8006812074bdac94e4f4de0ce3fa07e41c7bafb54d"} Jul 06 23:56:16.927251 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.926953 2558 scope.go:117] "RemoveContainer" containerID="b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845" Jul 06 23:56:16.936168 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.936143 2558 scope.go:117] "RemoveContainer" containerID="b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845" Jul 06 23:56:16.936478 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:56:16.936459 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845\": container with ID starting with b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845 not found: ID does not exist" containerID="b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845" Jul 06 23:56:16.936533 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.936486 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845"} err="failed to get container status \"b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845\": rpc error: code = NotFound desc = could not find container \"b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845\": container with ID starting with b4cf0295d3cb0f0c7fd0ef618a3f3753bd59074057ca951dd84a87f6ed156845 not found: ID does not exist" Jul 06 23:56:16.952631 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.952602 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb"] Jul 06 23:56:16.963104 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:16.963076 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vfskb"] Jul 06 23:56:18.050811 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:18.050778 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8592a4a2-a6be-4c7a-8a4f-8324d1c22569" path="/var/lib/kubelet/pods/8592a4a2-a6be-4c7a-8a4f-8324d1c22569/volumes" Jul 06 23:56:19.725789 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:19.725755 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:19.823664 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:19.823628 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:32.168652 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.168616 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc"] Jul 06 23:56:32.169133 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.168901 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8592a4a2-a6be-4c7a-8a4f-8324d1c22569" containerName="manager" Jul 06 23:56:32.169133 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.168913 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8592a4a2-a6be-4c7a-8a4f-8324d1c22569" containerName="manager" Jul 06 23:56:32.169133 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.168971 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8592a4a2-a6be-4c7a-8a4f-8324d1c22569" containerName="manager" Jul 06 23:56:32.170786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.170769 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.177678 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.177656 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2vtfx\"" Jul 06 23:56:32.197036 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.197001 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc"] Jul 06 23:56:32.207423 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.207393 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.207584 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.207437 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.207584 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.207463 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.207584 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.207480 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4xzq\" (UniqueName: \"kubernetes.io/projected/86ddbcaa-9010-4191-a96d-1218f5c44b33-kube-api-access-z4xzq\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.207584 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.207507 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.207584 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.207534 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86ddbcaa-9010-4191-a96d-1218f5c44b33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.207754 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.207601 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.207754 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.207626 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.207754 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.207641 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.215870 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.215819 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:32.308466 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308429 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.308466 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308471 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.308696 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308491 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.308696 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308616 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.308696 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308658 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.308696 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308689 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.308948 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308717 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4xzq\" (UniqueName: \"kubernetes.io/projected/86ddbcaa-9010-4191-a96d-1218f5c44b33-kube-api-access-z4xzq\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.308948 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308750 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.308948 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308796 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86ddbcaa-9010-4191-a96d-1218f5c44b33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.308948 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308905 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.309146 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308944 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.309146 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.308991 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.309222 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.309159 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.309449 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.309432 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86ddbcaa-9010-4191-a96d-1218f5c44b33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.311023 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.311000 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.311218 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.311201 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.320716 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.320692 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4xzq\" (UniqueName: \"kubernetes.io/projected/86ddbcaa-9010-4191-a96d-1218f5c44b33-kube-api-access-z4xzq\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.322394 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.322370 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86ddbcaa-9010-4191-a96d-1218f5c44b33-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-zh8gc\" (UID: \"86ddbcaa-9010-4191-a96d-1218f5c44b33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.338173 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.338150 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:32.480699 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.480657 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:32.617103 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.617075 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc"] Jul 06 23:56:32.619403 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:56:32.619375 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ddbcaa_9010_4191_a96d_1218f5c44b33.slice/crio-55091f5b6b5301941e5bbc55a0377df033654e660367711f52c5e36256acc5e0 WatchSource:0}: Error finding container 55091f5b6b5301941e5bbc55a0377df033654e660367711f52c5e36256acc5e0: Status 404 returned error can't find the container with id 55091f5b6b5301941e5bbc55a0377df033654e660367711f52c5e36256acc5e0 Jul 06 23:56:32.992117 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:32.992072 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" event={"ID":"86ddbcaa-9010-4191-a96d-1218f5c44b33","Type":"ContainerStarted","Data":"55091f5b6b5301941e5bbc55a0377df033654e660367711f52c5e36256acc5e0"} Jul 06 23:56:34.802362 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:34.802319 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 23:56:34.802661 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:34.802397 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 23:56:34.802661 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:34.802424 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 23:56:35.002273 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:35.002231 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" event={"ID":"86ddbcaa-9010-4191-a96d-1218f5c44b33","Type":"ContainerStarted","Data":"53c6a69090cde0e479491385c0d8f29c7f9032b8bc0d1c70ca70e283dc506eb8"} Jul 06 23:56:35.024056 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:35.023972 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" podStartSLOduration=0.843374425 podStartE2EDuration="3.023954897s" totalImagesPullingTime="2.180580472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:35.021829164 +0000 UTC m=+475.531990139" watchObservedRunningTime="2026-07-06 23:56:35.023954897 +0000 UTC m=+475.534115862" Jul 06 23:56:35.481769 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:35.481731 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:35.486520 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:35.486495 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:36.007871 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.007824 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:36.008767 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.008750 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:56:36.042242 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.042207 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plzfg"] Jul 06 23:56:36.044203 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.044164 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:36.044714 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.044689 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:36.047461 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.047439 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 23:56:36.047731 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.047715 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2kgkj\"" Jul 06 23:56:36.053701 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.053672 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plzfg"] Jul 06 23:56:36.138108 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.138071 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plzfg"] Jul 06 23:56:36.141623 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.141588 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:36.144127 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.144093 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcsn6\" (UniqueName: \"kubernetes.io/projected/f142d88d-12af-4b12-ad38-388a718be833-kube-api-access-lcsn6\") pod \"limitador-limitador-64c94fd64f-plzfg\" (UID: \"f142d88d-12af-4b12-ad38-388a718be833\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:36.144327 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.144304 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f142d88d-12af-4b12-ad38-388a718be833-config-file\") pod \"limitador-limitador-64c94fd64f-plzfg\" (UID: \"f142d88d-12af-4b12-ad38-388a718be833\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:36.199559 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.199523 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:36.244743 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.244706 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcsn6\" (UniqueName: \"kubernetes.io/projected/f142d88d-12af-4b12-ad38-388a718be833-kube-api-access-lcsn6\") pod \"limitador-limitador-64c94fd64f-plzfg\" (UID: \"f142d88d-12af-4b12-ad38-388a718be833\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:36.244939 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.244782 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f142d88d-12af-4b12-ad38-388a718be833-config-file\") pod \"limitador-limitador-64c94fd64f-plzfg\" (UID: \"f142d88d-12af-4b12-ad38-388a718be833\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:36.245378 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.245359 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f142d88d-12af-4b12-ad38-388a718be833-config-file\") pod \"limitador-limitador-64c94fd64f-plzfg\" (UID: \"f142d88d-12af-4b12-ad38-388a718be833\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:36.253218 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.253185 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcsn6\" (UniqueName: \"kubernetes.io/projected/f142d88d-12af-4b12-ad38-388a718be833-kube-api-access-lcsn6\") pod \"limitador-limitador-64c94fd64f-plzfg\" (UID: \"f142d88d-12af-4b12-ad38-388a718be833\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:36.358527 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.358435 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:36.494372 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.494210 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plzfg"] Jul 06 23:56:36.497243 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:56:36.497205 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf142d88d_12af_4b12_ad38_388a718be833.slice/crio-5333bfa9091fe2360dd10a9e80330d91ab4de7afce3bfead490ff0d0f6bb4872 WatchSource:0}: Error finding container 5333bfa9091fe2360dd10a9e80330d91ab4de7afce3bfead490ff0d0f6bb4872: Status 404 returned error can't find the container with id 5333bfa9091fe2360dd10a9e80330d91ab4de7afce3bfead490ff0d0f6bb4872 Jul 06 23:56:36.863457 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.863422 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:36.926102 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:36.926067 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:37.016764 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:37.016719 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" event={"ID":"f142d88d-12af-4b12-ad38-388a718be833","Type":"ContainerStarted","Data":"5333bfa9091fe2360dd10a9e80330d91ab4de7afce3bfead490ff0d0f6bb4872"} Jul 06 23:56:39.027379 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:39.027343 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" event={"ID":"f142d88d-12af-4b12-ad38-388a718be833","Type":"ContainerStarted","Data":"7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44"} Jul 06 23:56:39.027776 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:39.027491 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:39.045735 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:39.045689 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" podStartSLOduration=0.607832103 podStartE2EDuration="3.045675174s" totalImagesPullingTime="2.437843071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:39.044291906 +0000 UTC m=+479.554452878" watchObservedRunningTime="2026-07-06 23:56:39.045675174 +0000 UTC m=+479.555836131" Jul 06 23:56:50.033464 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:50.033428 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:52.135820 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:52.135780 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plzfg"] Jul 06 23:56:52.136238 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:52.136009 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" podUID="f142d88d-12af-4b12-ad38-388a718be833" containerName="limitador" containerID="cri-o://7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44" gracePeriod=30 Jul 06 23:56:52.679751 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:52.679725 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:52.789033 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:52.788939 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcsn6\" (UniqueName: \"kubernetes.io/projected/f142d88d-12af-4b12-ad38-388a718be833-kube-api-access-lcsn6\") pod \"f142d88d-12af-4b12-ad38-388a718be833\" (UID: \"f142d88d-12af-4b12-ad38-388a718be833\") " Jul 06 23:56:52.789033 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:52.789019 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f142d88d-12af-4b12-ad38-388a718be833-config-file\") pod \"f142d88d-12af-4b12-ad38-388a718be833\" (UID: \"f142d88d-12af-4b12-ad38-388a718be833\") " Jul 06 23:56:52.789382 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:52.789356 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f142d88d-12af-4b12-ad38-388a718be833-config-file" (OuterVolumeSpecName: "config-file") pod "f142d88d-12af-4b12-ad38-388a718be833" (UID: "f142d88d-12af-4b12-ad38-388a718be833"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:56:52.791238 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:52.791212 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f142d88d-12af-4b12-ad38-388a718be833-kube-api-access-lcsn6" (OuterVolumeSpecName: "kube-api-access-lcsn6") pod "f142d88d-12af-4b12-ad38-388a718be833" (UID: "f142d88d-12af-4b12-ad38-388a718be833"). InnerVolumeSpecName "kube-api-access-lcsn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:56:52.889984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:52.889944 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcsn6\" (UniqueName: \"kubernetes.io/projected/f142d88d-12af-4b12-ad38-388a718be833-kube-api-access-lcsn6\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:56:52.889984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:52.889977 2558 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f142d88d-12af-4b12-ad38-388a718be833-config-file\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:56:53.086760 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:53.086671 2558 generic.go:356] "Generic (PLEG): container finished" podID="f142d88d-12af-4b12-ad38-388a718be833" containerID="7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44" exitCode=0 Jul 06 23:56:53.086760 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:53.086719 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" Jul 06 23:56:53.087006 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:53.086717 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" event={"ID":"f142d88d-12af-4b12-ad38-388a718be833","Type":"ContainerDied","Data":"7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44"} Jul 06 23:56:53.087006 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:53.086899 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-plzfg" event={"ID":"f142d88d-12af-4b12-ad38-388a718be833","Type":"ContainerDied","Data":"5333bfa9091fe2360dd10a9e80330d91ab4de7afce3bfead490ff0d0f6bb4872"} Jul 06 23:56:53.087006 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:53.086939 2558 scope.go:117] "RemoveContainer" containerID="7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44" Jul 06 23:56:53.095950 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:53.095927 2558 scope.go:117] "RemoveContainer" containerID="7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44" Jul 06 23:56:53.096240 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:56:53.096218 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44\": container with ID starting with 7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44 not found: ID does not exist" containerID="7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44" Jul 06 23:56:53.096312 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:53.096252 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44"} err="failed to get container status \"7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44\": rpc error: code = NotFound desc = could not find container \"7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44\": container with ID starting with 7f3254598d5e6afdccf9b4e815d9b51d99b81535a9a6d40f2b548ffa61aada44 not found: ID does not exist" Jul 06 23:56:53.108197 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:53.108163 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plzfg"] Jul 06 23:56:53.111385 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:53.111361 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-plzfg"] Jul 06 23:56:54.050522 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:54.050478 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f142d88d-12af-4b12-ad38-388a718be833" path="/var/lib/kubelet/pods/f142d88d-12af-4b12-ad38-388a718be833/volumes" Jul 06 23:56:57.616913 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.616879 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-rpsqs"] Jul 06 23:56:57.617392 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.617197 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f142d88d-12af-4b12-ad38-388a718be833" containerName="limitador" Jul 06 23:56:57.617392 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.617211 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f142d88d-12af-4b12-ad38-388a718be833" containerName="limitador" Jul 06 23:56:57.617392 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.617261 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f142d88d-12af-4b12-ad38-388a718be833" containerName="limitador" Jul 06 23:56:57.619750 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.619730 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-rpsqs" Jul 06 23:56:57.622240 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.622217 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-28mvd\"" Jul 06 23:56:57.622375 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.622266 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 23:56:57.629763 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.629736 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-rpsqs"] Jul 06 23:56:57.653882 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.653827 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:57.725580 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.725544 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frmfc\" (UniqueName: \"kubernetes.io/projected/dacb6431-f5d1-4e25-a6ac-2ae78d6813eb-kube-api-access-frmfc\") pod \"postgres-69fb4594cf-rpsqs\" (UID: \"dacb6431-f5d1-4e25-a6ac-2ae78d6813eb\") " pod="opendatahub/postgres-69fb4594cf-rpsqs" Jul 06 23:56:57.725741 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.725586 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dacb6431-f5d1-4e25-a6ac-2ae78d6813eb-data\") pod \"postgres-69fb4594cf-rpsqs\" (UID: \"dacb6431-f5d1-4e25-a6ac-2ae78d6813eb\") " pod="opendatahub/postgres-69fb4594cf-rpsqs" Jul 06 23:56:57.826720 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.826678 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frmfc\" (UniqueName: \"kubernetes.io/projected/dacb6431-f5d1-4e25-a6ac-2ae78d6813eb-kube-api-access-frmfc\") pod \"postgres-69fb4594cf-rpsqs\" (UID: \"dacb6431-f5d1-4e25-a6ac-2ae78d6813eb\") " pod="opendatahub/postgres-69fb4594cf-rpsqs" Jul 06 23:56:57.826923 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.826728 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dacb6431-f5d1-4e25-a6ac-2ae78d6813eb-data\") pod \"postgres-69fb4594cf-rpsqs\" (UID: \"dacb6431-f5d1-4e25-a6ac-2ae78d6813eb\") " pod="opendatahub/postgres-69fb4594cf-rpsqs" Jul 06 23:56:57.827143 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.827125 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dacb6431-f5d1-4e25-a6ac-2ae78d6813eb-data\") pod \"postgres-69fb4594cf-rpsqs\" (UID: \"dacb6431-f5d1-4e25-a6ac-2ae78d6813eb\") " pod="opendatahub/postgres-69fb4594cf-rpsqs" Jul 06 23:56:57.835665 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.835634 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frmfc\" (UniqueName: \"kubernetes.io/projected/dacb6431-f5d1-4e25-a6ac-2ae78d6813eb-kube-api-access-frmfc\") pod \"postgres-69fb4594cf-rpsqs\" (UID: \"dacb6431-f5d1-4e25-a6ac-2ae78d6813eb\") " pod="opendatahub/postgres-69fb4594cf-rpsqs" Jul 06 23:56:57.930755 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:57.930720 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-rpsqs" Jul 06 23:56:58.061385 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:58.061223 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-rpsqs"] Jul 06 23:56:58.064140 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:56:58.064113 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddacb6431_f5d1_4e25_a6ac_2ae78d6813eb.slice/crio-a1ab2394174ece8f7b1c1b5ad677ba8e8ec1ce46bd7c0e9ce45f034f998af7fd WatchSource:0}: Error finding container a1ab2394174ece8f7b1c1b5ad677ba8e8ec1ce46bd7c0e9ce45f034f998af7fd: Status 404 returned error can't find the container with id a1ab2394174ece8f7b1c1b5ad677ba8e8ec1ce46bd7c0e9ce45f034f998af7fd Jul 06 23:56:58.109271 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:56:58.109236 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-rpsqs" event={"ID":"dacb6431-f5d1-4e25-a6ac-2ae78d6813eb","Type":"ContainerStarted","Data":"a1ab2394174ece8f7b1c1b5ad677ba8e8ec1ce46bd7c0e9ce45f034f998af7fd"} Jul 06 23:57:03.141654 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:03.141555 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-rpsqs" event={"ID":"dacb6431-f5d1-4e25-a6ac-2ae78d6813eb","Type":"ContainerStarted","Data":"181bf0f49a89c9b06507c45ae26cc4f957de75061ef5d2195d86a3c8f2e984d5"} Jul 06 23:57:03.142089 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:03.141654 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-rpsqs" Jul 06 23:57:03.159676 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:03.159625 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-rpsqs" podStartSLOduration=1.46718044 podStartE2EDuration="6.159607151s" totalImagesPullingTime="4.692426711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:03.158236699 +0000 UTC m=+503.668397652" watchObservedRunningTime="2026-07-06 23:57:03.159607151 +0000 UTC m=+503.669768107" Jul 06 23:57:09.176757 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:09.176731 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-rpsqs" Jul 06 23:57:09.503401 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:09.503314 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:09.603419 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:09.603378 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:11.545300 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:11.545263 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:11.609647 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:11.609603 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:16.504149 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.504105 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:16.522480 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.522446 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d58f6997c-k47rt"] Jul 06 23:57:16.528525 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.528498 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:16.531397 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.531367 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 23:57:16.531534 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.531373 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nw684\"" Jul 06 23:57:16.532689 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.532670 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 23:57:16.540786 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.540762 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d58f6997c-k47rt"] Jul 06 23:57:16.571135 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.571096 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwjhc\" (UniqueName: \"kubernetes.io/projected/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-kube-api-access-vwjhc\") pod \"maas-controller-d58f6997c-k47rt\" (UID: \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\") " pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:16.571135 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.571137 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-webhook-cert\") pod \"maas-controller-d58f6997c-k47rt\" (UID: \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\") " pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:16.623979 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.623934 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:16.671867 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.671813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwjhc\" (UniqueName: \"kubernetes.io/projected/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-kube-api-access-vwjhc\") pod \"maas-controller-d58f6997c-k47rt\" (UID: \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\") " pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:16.672063 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.671903 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-webhook-cert\") pod \"maas-controller-d58f6997c-k47rt\" (UID: \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\") " pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:16.672063 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:57:16.672043 2558 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 23:57:16.672137 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:57:16.672108 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-webhook-cert podName:9b5d417e-6e92-4e4e-80ce-2f3ca457d63c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:17.172091078 +0000 UTC m=+517.682252014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-webhook-cert") pod "maas-controller-d58f6997c-k47rt" (UID: "9b5d417e-6e92-4e4e-80ce-2f3ca457d63c") : secret "maas-controller-webhook-cert" not found Jul 06 23:57:16.682036 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.682011 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwjhc\" (UniqueName: \"kubernetes.io/projected/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-kube-api-access-vwjhc\") pod \"maas-controller-d58f6997c-k47rt\" (UID: \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\") " pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:16.721502 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.721458 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d86fcfb77-mxz8j"] Jul 06 23:57:16.722425 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.722394 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:16.724451 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.724434 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 06 23:57:16.738710 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.738685 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d86fcfb77-mxz8j"] Jul 06 23:57:16.773181 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.773085 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld7hm\" (UniqueName: \"kubernetes.io/projected/7cd2be0a-44e9-4f87-9694-8da28b325194-kube-api-access-ld7hm\") pod \"maas-controller-6d86fcfb77-mxz8j\" (UID: \"7cd2be0a-44e9-4f87-9694-8da28b325194\") " pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 06 23:57:16.773181 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.773145 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cd2be0a-44e9-4f87-9694-8da28b325194-webhook-cert\") pod \"maas-controller-6d86fcfb77-mxz8j\" (UID: \"7cd2be0a-44e9-4f87-9694-8da28b325194\") " pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 06 23:57:16.867525 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.867483 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:16.874367 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.874334 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cd2be0a-44e9-4f87-9694-8da28b325194-webhook-cert\") pod \"maas-controller-6d86fcfb77-mxz8j\" (UID: \"7cd2be0a-44e9-4f87-9694-8da28b325194\") " pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 06 23:57:16.874539 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.874419 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld7hm\" (UniqueName: \"kubernetes.io/projected/7cd2be0a-44e9-4f87-9694-8da28b325194-kube-api-access-ld7hm\") pod \"maas-controller-6d86fcfb77-mxz8j\" (UID: \"7cd2be0a-44e9-4f87-9694-8da28b325194\") " pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 06 23:57:16.876984 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.876959 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cd2be0a-44e9-4f87-9694-8da28b325194-webhook-cert\") pod \"maas-controller-6d86fcfb77-mxz8j\" (UID: \"7cd2be0a-44e9-4f87-9694-8da28b325194\") " pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 06 23:57:16.883684 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:16.883659 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld7hm\" (UniqueName: \"kubernetes.io/projected/7cd2be0a-44e9-4f87-9694-8da28b325194-kube-api-access-ld7hm\") pod \"maas-controller-6d86fcfb77-mxz8j\" (UID: \"7cd2be0a-44e9-4f87-9694-8da28b325194\") " pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 06 23:57:17.034878 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:17.034757 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 06 23:57:17.168918 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:17.168737 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d86fcfb77-mxz8j"] Jul 06 23:57:17.171410 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:57:17.171385 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd2be0a_44e9_4f87_9694_8da28b325194.slice/crio-67a85ad0bfc1887d3a8a2f5e9247b06234fc681dad3e0f9ea76095974da9d5b1 WatchSource:0}: Error finding container 67a85ad0bfc1887d3a8a2f5e9247b06234fc681dad3e0f9ea76095974da9d5b1: Status 404 returned error can't find the container with id 67a85ad0bfc1887d3a8a2f5e9247b06234fc681dad3e0f9ea76095974da9d5b1 Jul 06 23:57:17.176668 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:17.176643 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-webhook-cert\") pod \"maas-controller-d58f6997c-k47rt\" (UID: \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\") " pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:17.179150 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:17.179127 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-webhook-cert\") pod \"maas-controller-d58f6997c-k47rt\" (UID: \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\") " pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:17.200572 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:17.200542 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" event={"ID":"7cd2be0a-44e9-4f87-9694-8da28b325194","Type":"ContainerStarted","Data":"67a85ad0bfc1887d3a8a2f5e9247b06234fc681dad3e0f9ea76095974da9d5b1"} Jul 06 23:57:17.438779 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:17.438738 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:17.573522 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:17.573490 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d58f6997c-k47rt"] Jul 06 23:57:17.576548 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:57:17.576519 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5d417e_6e92_4e4e_80ce_2f3ca457d63c.slice/crio-3bc617127b84e8dcb25925b1650b52110949e4a086d39a5d6370b4edfdf95739 WatchSource:0}: Error finding container 3bc617127b84e8dcb25925b1650b52110949e4a086d39a5d6370b4edfdf95739: Status 404 returned error can't find the container with id 3bc617127b84e8dcb25925b1650b52110949e4a086d39a5d6370b4edfdf95739 Jul 06 23:57:18.208206 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:18.208163 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d58f6997c-k47rt" event={"ID":"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c","Type":"ContainerStarted","Data":"3bc617127b84e8dcb25925b1650b52110949e4a086d39a5d6370b4edfdf95739"} Jul 06 23:57:19.774143 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:19.774047 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:19.867961 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:19.867884 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:20.219420 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:20.219380 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" event={"ID":"7cd2be0a-44e9-4f87-9694-8da28b325194","Type":"ContainerStarted","Data":"3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab"} Jul 06 23:57:20.219686 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:20.219497 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 06 23:57:20.221065 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:20.221036 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d58f6997c-k47rt" event={"ID":"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c","Type":"ContainerStarted","Data":"6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2"} Jul 06 23:57:20.221183 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:20.221088 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:20.238522 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:20.238472 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" podStartSLOduration=1.906680465 podStartE2EDuration="4.238454768s" totalImagesPullingTime="2.331774303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:20.237540343 +0000 UTC m=+520.747701300" watchObservedRunningTime="2026-07-06 23:57:20.238454768 +0000 UTC m=+520.748615724" Jul 06 23:57:20.259677 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:20.259619 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d58f6997c-k47rt" podStartSLOduration=2.321876314 podStartE2EDuration="4.259601564s" totalImagesPullingTime="1.93772525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:20.258994071 +0000 UTC m=+520.769155029" watchObservedRunningTime="2026-07-06 23:57:20.259601564 +0000 UTC m=+520.769762520" Jul 06 23:57:31.230799 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.230711 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 06 23:57:31.231299 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.231023 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:31.304046 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.304016 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d58f6997c-k47rt"] Jul 06 23:57:31.304214 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.304184 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-d58f6997c-k47rt" podUID="9b5d417e-6e92-4e4e-80ce-2f3ca457d63c" containerName="manager" containerID="cri-o://6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2" gracePeriod=30 Jul 06 23:57:31.581143 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.581112 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:31.699124 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.699090 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-webhook-cert\") pod \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\" (UID: \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\") " Jul 06 23:57:31.699304 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.699137 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwjhc\" (UniqueName: \"kubernetes.io/projected/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-kube-api-access-vwjhc\") pod \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\" (UID: \"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c\") " Jul 06 23:57:31.701504 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.701473 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-kube-api-access-vwjhc" (OuterVolumeSpecName: "kube-api-access-vwjhc") pod "9b5d417e-6e92-4e4e-80ce-2f3ca457d63c" (UID: "9b5d417e-6e92-4e4e-80ce-2f3ca457d63c"). InnerVolumeSpecName "kube-api-access-vwjhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:57:31.701615 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.701530 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9b5d417e-6e92-4e4e-80ce-2f3ca457d63c" (UID: "9b5d417e-6e92-4e4e-80ce-2f3ca457d63c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:57:31.800219 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.800121 2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-webhook-cert\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:57:31.800219 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:31.800154 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwjhc\" (UniqueName: \"kubernetes.io/projected/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c-kube-api-access-vwjhc\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 06 23:57:32.270815 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:32.270781 2558 generic.go:356] "Generic (PLEG): container finished" podID="9b5d417e-6e92-4e4e-80ce-2f3ca457d63c" containerID="6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2" exitCode=0 Jul 06 23:57:32.271235 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:32.270831 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d58f6997c-k47rt" Jul 06 23:57:32.271235 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:32.270880 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d58f6997c-k47rt" event={"ID":"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c","Type":"ContainerDied","Data":"6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2"} Jul 06 23:57:32.271235 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:32.270931 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d58f6997c-k47rt" event={"ID":"9b5d417e-6e92-4e4e-80ce-2f3ca457d63c","Type":"ContainerDied","Data":"3bc617127b84e8dcb25925b1650b52110949e4a086d39a5d6370b4edfdf95739"} Jul 06 23:57:32.271235 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:32.270954 2558 scope.go:117] "RemoveContainer" containerID="6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2" Jul 06 23:57:32.280888 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:32.280819 2558 scope.go:117] "RemoveContainer" containerID="6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2" Jul 06 23:57:32.281144 ip-10-0-130-213 kubenswrapper[2558]: E0706 23:57:32.281124 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2\": container with ID starting with 6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2 not found: ID does not exist" containerID="6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2" Jul 06 23:57:32.281205 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:32.281156 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2"} err="failed to get container status \"6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2\": rpc error: code = NotFound desc = could not find container \"6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2\": container with ID starting with 6bae654215d981aff5415eadc50d3ff66d2b8fe5f5e4a7ab1d1f0f0884ccdab2 not found: ID does not exist" Jul 06 23:57:32.291934 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:32.290352 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d58f6997c-k47rt"] Jul 06 23:57:32.297127 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:32.297099 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d58f6997c-k47rt"] Jul 06 23:57:34.050780 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:34.050743 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b5d417e-6e92-4e4e-80ce-2f3ca457d63c" path="/var/lib/kubelet/pods/9b5d417e-6e92-4e4e-80ce-2f3ca457d63c/volumes" Jul 06 23:57:49.441003 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:49.440963 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:49.978587 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:49.978541 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:50.015907 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.015878 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6b58c8fc8d-bfggs"] Jul 06 23:57:50.016194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.016181 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b5d417e-6e92-4e4e-80ce-2f3ca457d63c" containerName="manager" Jul 06 23:57:50.016237 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.016196 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b5d417e-6e92-4e4e-80ce-2f3ca457d63c" containerName="manager" Jul 06 23:57:50.016270 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.016260 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b5d417e-6e92-4e4e-80ce-2f3ca457d63c" containerName="manager" Jul 06 23:57:50.023113 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.023090 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.025978 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.025953 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 23:57:50.026094 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.025999 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5mk6f\"" Jul 06 23:57:50.031455 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.031427 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6b58c8fc8d-bfggs"] Jul 06 23:57:50.154936 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.154899 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8knf\" (UniqueName: \"kubernetes.io/projected/7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c-kube-api-access-j8knf\") pod \"maas-api-6b58c8fc8d-bfggs\" (UID: \"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c\") " pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.155107 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.154982 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c-maas-api-tls\") pod \"maas-api-6b58c8fc8d-bfggs\" (UID: \"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c\") " pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.155107 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.155008 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c-openshift-service-ca\") pod \"maas-api-6b58c8fc8d-bfggs\" (UID: \"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c\") " pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.255686 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.255579 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c-maas-api-tls\") pod \"maas-api-6b58c8fc8d-bfggs\" (UID: \"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c\") " pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.255686 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.255639 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c-openshift-service-ca\") pod \"maas-api-6b58c8fc8d-bfggs\" (UID: \"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c\") " pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.255686 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.255682 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8knf\" (UniqueName: \"kubernetes.io/projected/7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c-kube-api-access-j8knf\") pod \"maas-api-6b58c8fc8d-bfggs\" (UID: \"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c\") " pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.256589 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.256517 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c-openshift-service-ca\") pod \"maas-api-6b58c8fc8d-bfggs\" (UID: \"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c\") " pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.258668 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.258639 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c-maas-api-tls\") pod \"maas-api-6b58c8fc8d-bfggs\" (UID: \"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c\") " pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.265098 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.265059 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8knf\" (UniqueName: \"kubernetes.io/projected/7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c-kube-api-access-j8knf\") pod \"maas-api-6b58c8fc8d-bfggs\" (UID: \"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c\") " pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.298817 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.298779 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:50.334770 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.334738 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:50.418808 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.418772 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:50.488916 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.487636 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6b58c8fc8d-bfggs"] Jul 06 23:57:50.526919 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.526824 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:50.548249 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.548217 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5trzj"] Jul 06 23:57:50.550860 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.550832 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" Jul 06 23:57:50.553562 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.553538 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 23:57:50.567008 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.566976 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5trzj"] Jul 06 23:57:50.571976 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.571949 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-wwn72"] Jul 06 23:57:50.573963 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.573948 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-wwn72" Jul 06 23:57:50.576725 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.576706 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wx44d\"" Jul 06 23:57:50.582742 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.582719 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-wwn72"] Jul 06 23:57:50.658235 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.658195 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44a3bbc-40b6-417a-b1fe-1bd13330ed47-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5trzj\" (UID: \"d44a3bbc-40b6-417a-b1fe-1bd13330ed47\") " pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" Jul 06 23:57:50.658401 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.658251 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d27bb777-193f-4ee0-9117-3c0d57eea737-plugins-config-volume\") pod \"payload-processing-58f86c7798-wwn72\" (UID: \"d27bb777-193f-4ee0-9117-3c0d57eea737\") " pod="openshift-ingress/payload-processing-58f86c7798-wwn72" Jul 06 23:57:50.658401 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.658297 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-885g2\" (UniqueName: \"kubernetes.io/projected/d27bb777-193f-4ee0-9117-3c0d57eea737-kube-api-access-885g2\") pod \"payload-processing-58f86c7798-wwn72\" (UID: \"d27bb777-193f-4ee0-9117-3c0d57eea737\") " pod="openshift-ingress/payload-processing-58f86c7798-wwn72" Jul 06 23:57:50.658401 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.658335 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68nns\" (UniqueName: \"kubernetes.io/projected/d44a3bbc-40b6-417a-b1fe-1bd13330ed47-kube-api-access-68nns\") pod \"payload-pre-processing-7964457696-5trzj\" (UID: \"d44a3bbc-40b6-417a-b1fe-1bd13330ed47\") " pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" Jul 06 23:57:50.759320 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.759283 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44a3bbc-40b6-417a-b1fe-1bd13330ed47-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5trzj\" (UID: \"d44a3bbc-40b6-417a-b1fe-1bd13330ed47\") " pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" Jul 06 23:57:50.759514 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.759335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d27bb777-193f-4ee0-9117-3c0d57eea737-plugins-config-volume\") pod \"payload-processing-58f86c7798-wwn72\" (UID: \"d27bb777-193f-4ee0-9117-3c0d57eea737\") " pod="openshift-ingress/payload-processing-58f86c7798-wwn72" Jul 06 23:57:50.759514 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.759354 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-885g2\" (UniqueName: \"kubernetes.io/projected/d27bb777-193f-4ee0-9117-3c0d57eea737-kube-api-access-885g2\") pod \"payload-processing-58f86c7798-wwn72\" (UID: \"d27bb777-193f-4ee0-9117-3c0d57eea737\") " pod="openshift-ingress/payload-processing-58f86c7798-wwn72" Jul 06 23:57:50.759514 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.759383 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68nns\" (UniqueName: \"kubernetes.io/projected/d44a3bbc-40b6-417a-b1fe-1bd13330ed47-kube-api-access-68nns\") pod \"payload-pre-processing-7964457696-5trzj\" (UID: \"d44a3bbc-40b6-417a-b1fe-1bd13330ed47\") " pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" Jul 06 23:57:50.760200 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.760165 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44a3bbc-40b6-417a-b1fe-1bd13330ed47-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5trzj\" (UID: \"d44a3bbc-40b6-417a-b1fe-1bd13330ed47\") " pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" Jul 06 23:57:50.760313 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.760172 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d27bb777-193f-4ee0-9117-3c0d57eea737-plugins-config-volume\") pod \"payload-processing-58f86c7798-wwn72\" (UID: \"d27bb777-193f-4ee0-9117-3c0d57eea737\") " pod="openshift-ingress/payload-processing-58f86c7798-wwn72" Jul 06 23:57:50.769255 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.769222 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-885g2\" (UniqueName: \"kubernetes.io/projected/d27bb777-193f-4ee0-9117-3c0d57eea737-kube-api-access-885g2\") pod \"payload-processing-58f86c7798-wwn72\" (UID: \"d27bb777-193f-4ee0-9117-3c0d57eea737\") " pod="openshift-ingress/payload-processing-58f86c7798-wwn72" Jul 06 23:57:50.769372 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.769278 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68nns\" (UniqueName: \"kubernetes.io/projected/d44a3bbc-40b6-417a-b1fe-1bd13330ed47-kube-api-access-68nns\") pod \"payload-pre-processing-7964457696-5trzj\" (UID: \"d44a3bbc-40b6-417a-b1fe-1bd13330ed47\") " pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" Jul 06 23:57:50.861383 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.861284 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" Jul 06 23:57:50.884785 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:50.884747 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-wwn72" Jul 06 23:57:51.013294 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:51.013265 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5trzj"] Jul 06 23:57:51.015521 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:57:51.015476 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44a3bbc_40b6_417a_b1fe_1bd13330ed47.slice/crio-60c7b91daeda739067b280fd8e1674a742a6b176295efa5ccc7cf9ce09512de2 WatchSource:0}: Error finding container 60c7b91daeda739067b280fd8e1674a742a6b176295efa5ccc7cf9ce09512de2: Status 404 returned error can't find the container with id 60c7b91daeda739067b280fd8e1674a742a6b176295efa5ccc7cf9ce09512de2 Jul 06 23:57:51.035134 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:51.035102 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-wwn72"] Jul 06 23:57:51.037037 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:57:51.036997 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd27bb777_193f_4ee0_9117_3c0d57eea737.slice/crio-57aeec4073b339f70a7d6bd2c99a62d238215f8120fe923498a135621149d6d5 WatchSource:0}: Error finding container 57aeec4073b339f70a7d6bd2c99a62d238215f8120fe923498a135621149d6d5: Status 404 returned error can't find the container with id 57aeec4073b339f70a7d6bd2c99a62d238215f8120fe923498a135621149d6d5 Jul 06 23:57:51.302781 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:51.302713 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:51.352292 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:51.352249 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" event={"ID":"d44a3bbc-40b6-417a-b1fe-1bd13330ed47","Type":"ContainerStarted","Data":"60c7b91daeda739067b280fd8e1674a742a6b176295efa5ccc7cf9ce09512de2"} Jul 06 23:57:51.353792 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:51.353751 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-wwn72" event={"ID":"d27bb777-193f-4ee0-9117-3c0d57eea737","Type":"ContainerStarted","Data":"57aeec4073b339f70a7d6bd2c99a62d238215f8120fe923498a135621149d6d5"} Jul 06 23:57:51.355064 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:51.355035 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6b58c8fc8d-bfggs" event={"ID":"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c","Type":"ContainerStarted","Data":"c3517f94bc2af17fb62fc8c8b8b821553677750dcbc0399b874ec5fb79f6ae1b"} Jul 06 23:57:53.368476 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:53.368435 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" event={"ID":"d44a3bbc-40b6-417a-b1fe-1bd13330ed47","Type":"ContainerStarted","Data":"3a43e561242980c47e032205c01e6d3d610250e5adf4ef4b82969c882c8ae9a4"} Jul 06 23:57:53.368991 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:53.368509 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" Jul 06 23:57:53.370031 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:53.369996 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-wwn72" event={"ID":"d27bb777-193f-4ee0-9117-3c0d57eea737","Type":"ContainerStarted","Data":"c7e004808e823a33ca1df2cca12583c49ea72d55e3597043a993ae377717a02b"} Jul 06 23:57:53.370169 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:53.370086 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-wwn72" Jul 06 23:57:53.371563 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:53.371539 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6b58c8fc8d-bfggs" event={"ID":"7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c","Type":"ContainerStarted","Data":"50295e07037febd25ea0526bc6f15398cd9be6511400b730f85896acacb3a375"} Jul 06 23:57:53.371695 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:53.371620 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:53.385764 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:53.385698 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" podStartSLOduration=1.5914894739999998 podStartE2EDuration="3.385670857s" totalImagesPullingTime="1.794181383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:53.384453551 +0000 UTC m=+553.894614521" watchObservedRunningTime="2026-07-06 23:57:53.385670857 +0000 UTC m=+553.895831817" Jul 06 23:57:53.403165 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:53.402206 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-wwn72" podStartSLOduration=1.632004567 podStartE2EDuration="3.402182849s" totalImagesPullingTime="1.770178282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:53.39947036 +0000 UTC m=+553.909631319" watchObservedRunningTime="2026-07-06 23:57:53.402182849 +0000 UTC m=+553.912343807" Jul 06 23:57:53.417418 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:53.417364 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6b58c8fc8d-bfggs" podStartSLOduration=2.101595003 podStartE2EDuration="4.417346212s" totalImagesPullingTime="2.315751209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:53.415262069 +0000 UTC m=+553.925423027" watchObservedRunningTime="2026-07-06 23:57:53.417346212 +0000 UTC m=+553.927507169" Jul 06 23:57:58.393508 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:58.393476 2558 generic.go:356] "Generic (PLEG): container finished" podID="86ddbcaa-9010-4191-a96d-1218f5c44b33" containerID="53c6a69090cde0e479491385c0d8f29c7f9032b8bc0d1c70ca70e283dc506eb8" exitCode=0 Jul 06 23:57:58.394064 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:58.393559 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" event={"ID":"86ddbcaa-9010-4191-a96d-1218f5c44b33","Type":"ContainerDied","Data":"53c6a69090cde0e479491385c0d8f29c7f9032b8bc0d1c70ca70e283dc506eb8"} Jul 06 23:57:58.394064 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:58.394046 2558 scope.go:117] "RemoveContainer" containerID="53c6a69090cde0e479491385c0d8f29c7f9032b8bc0d1c70ca70e283dc506eb8" Jul 06 23:57:58.395098 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:58.395066 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 23:57:58.395166 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:58.395143 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 23:57:58.395202 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:58.395175 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 23:57:59.384713 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:59.384669 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6b58c8fc8d-bfggs" Jul 06 23:57:59.408619 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:59.408578 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" event={"ID":"86ddbcaa-9010-4191-a96d-1218f5c44b33","Type":"ContainerStarted","Data":"684db056b0869d530e002c18bc0b3aa17c7c975f870cb25b365b5cda3af25f7b"} Jul 06 23:57:59.492877 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:59.489329 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:57:59.492877 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:59.489385 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zh8gc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body= Jul 06 23:57:59.492877 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:57:59.489462 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" podUID="86ddbcaa-9010-4191-a96d-1218f5c44b33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" Jul 06 23:58:00.370210 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:00.370175 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:58:00.475991 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:00.475952 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:58:00.482102 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:00.482065 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zh8gc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body= Jul 06 23:58:00.482280 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:00.482134 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" podUID="86ddbcaa-9010-4191-a96d-1218f5c44b33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" Jul 06 23:58:01.481470 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:01.481432 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zh8gc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body= Jul 06 23:58:01.482114 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:01.482080 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" podUID="86ddbcaa-9010-4191-a96d-1218f5c44b33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" Jul 06 23:58:02.481297 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:02.481259 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zh8gc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body= Jul 06 23:58:02.481709 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:02.481672 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" podUID="86ddbcaa-9010-4191-a96d-1218f5c44b33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" Jul 06 23:58:03.485258 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:03.485220 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:58:03.485664 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:03.485601 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:58:03.486383 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:03.486364 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zh8gc" Jul 06 23:58:04.378529 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:04.378500 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-wwn72" Jul 06 23:58:04.378752 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:04.378567 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-5trzj" Jul 06 23:58:07.878554 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.878522 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f"] Jul 06 23:58:07.880953 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.880930 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:07.883870 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.883817 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 23:58:07.883985 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.883923 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 06 23:58:07.884046 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.883986 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sb2d5\"" Jul 06 23:58:07.885199 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.885172 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 23:58:07.894677 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.894653 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f"] Jul 06 23:58:07.914061 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.914028 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:07.914239 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.914162 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh9fg\" (UniqueName: \"kubernetes.io/projected/23d4b580-7008-4b78-ab1f-63f4ac558f83-kube-api-access-nh9fg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:07.914239 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.914200 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:07.914239 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.914222 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d4b580-7008-4b78-ab1f-63f4ac558f83-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:07.914400 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.914261 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:07.914400 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.914306 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:07.924983 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:07.924949 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:58:08.015088 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.015056 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.015249 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.015117 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.015249 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.015147 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh9fg\" (UniqueName: \"kubernetes.io/projected/23d4b580-7008-4b78-ab1f-63f4ac558f83-kube-api-access-nh9fg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.015249 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.015167 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.015249 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.015181 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d4b580-7008-4b78-ab1f-63f4ac558f83-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.015249 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.015206 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.015549 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.015524 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.015687 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.015559 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.015687 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.015604 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.017597 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.017572 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d4b580-7008-4b78-ab1f-63f4ac558f83-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.017840 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.017825 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d4b580-7008-4b78-ab1f-63f4ac558f83-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.023924 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.023906 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh9fg\" (UniqueName: \"kubernetes.io/projected/23d4b580-7008-4b78-ab1f-63f4ac558f83-kube-api-access-nh9fg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f\" (UID: \"23d4b580-7008-4b78-ab1f-63f4ac558f83\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.192149 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.192119 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:08.349670 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.349514 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f"] Jul 06 23:58:08.352268 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:58:08.352238 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d4b580_7008_4b78_ab1f_63f4ac558f83.slice/crio-b60b36d23df8be7b849f83cdc6c943b15426e3a5952d4c231d541bf1fb1e8228 WatchSource:0}: Error finding container b60b36d23df8be7b849f83cdc6c943b15426e3a5952d4c231d541bf1fb1e8228: Status 404 returned error can't find the container with id b60b36d23df8be7b849f83cdc6c943b15426e3a5952d4c231d541bf1fb1e8228 Jul 06 23:58:08.451460 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:08.451375 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" event={"ID":"23d4b580-7008-4b78-ab1f-63f4ac558f83","Type":"ContainerStarted","Data":"b60b36d23df8be7b849f83cdc6c943b15426e3a5952d4c231d541bf1fb1e8228"} Jul 06 23:58:09.457841 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:09.457808 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" event={"ID":"23d4b580-7008-4b78-ab1f-63f4ac558f83","Type":"ContainerStarted","Data":"40785809fb99d637b4a5b616d7ce9782d59bbe4e4a5ce2bc03a8c03795ee4c0f"} Jul 06 23:58:09.476565 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:09.476510 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" podStartSLOduration=1.480152274 podStartE2EDuration="2.476485274s" totalImagesPullingTime="996.333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:09.475130642 +0000 UTC m=+569.985291625" watchObservedRunningTime="2026-07-06 23:58:09.476485274 +0000 UTC m=+569.986646228" Jul 06 23:58:18.192246 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:18.192207 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:18.192885 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:18.192305 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:18.205100 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:18.205073 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:18.507916 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:18.507807 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f" Jul 06 23:58:22.117543 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:22.117491 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:58:39.970252 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:39.970223 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 06 23:58:39.975606 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:39.975576 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 06 23:58:49.012194 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.012160 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq"] Jul 06 23:58:49.015274 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.015256 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.018893 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.018871 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 23:58:49.028186 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.028158 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq"] Jul 06 23:58:49.047398 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.047359 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:58:49.151518 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.151490 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.151518 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.151522 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2587b7a8-36b1-4f10-b724-23dcc98fbafc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.151737 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.151550 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x2k2\" (UniqueName: \"kubernetes.io/projected/2587b7a8-36b1-4f10-b724-23dcc98fbafc-kube-api-access-6x2k2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.151737 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.151569 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.151737 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.151722 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.151844 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.151774 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.253061 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.253023 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.253285 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.253129 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.253285 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.253160 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.253285 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.253202 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.253285 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.253231 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2587b7a8-36b1-4f10-b724-23dcc98fbafc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.253285 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.253268 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x2k2\" (UniqueName: \"kubernetes.io/projected/2587b7a8-36b1-4f10-b724-23dcc98fbafc-kube-api-access-6x2k2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.253609 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.253580 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.253654 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.253613 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.253654 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.253644 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.255548 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.255526 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2587b7a8-36b1-4f10-b724-23dcc98fbafc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.255801 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.255780 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2587b7a8-36b1-4f10-b724-23dcc98fbafc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.262039 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.262003 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x2k2\" (UniqueName: \"kubernetes.io/projected/2587b7a8-36b1-4f10-b724-23dcc98fbafc-kube-api-access-6x2k2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnrtq\" (UID: \"2587b7a8-36b1-4f10-b724-23dcc98fbafc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.324979 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.324900 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:49.461410 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.461385 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq"] Jul 06 23:58:49.464172 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:58:49.464145 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2587b7a8_36b1_4f10_b724_23dcc98fbafc.slice/crio-606023017edc3365ef27b6722661c57734e44562a9345f157b6a703d799c62c4 WatchSource:0}: Error finding container 606023017edc3365ef27b6722661c57734e44562a9345f157b6a703d799c62c4: Status 404 returned error can't find the container with id 606023017edc3365ef27b6722661c57734e44562a9345f157b6a703d799c62c4 Jul 06 23:58:49.465539 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.465519 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:58:49.630708 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:49.630678 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" event={"ID":"2587b7a8-36b1-4f10-b724-23dcc98fbafc","Type":"ContainerStarted","Data":"606023017edc3365ef27b6722661c57734e44562a9345f157b6a703d799c62c4"} Jul 06 23:58:50.636566 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:50.636530 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" event={"ID":"2587b7a8-36b1-4f10-b724-23dcc98fbafc","Type":"ContainerStarted","Data":"c6d720f2e134787bf483526bc0c6eedca0a823012f5406a9787e7ced3a05a742"} Jul 06 23:58:50.663754 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:50.663701 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" podStartSLOduration=2.521677171 podStartE2EDuration="2.663683137s" totalImagesPullingTime="142.005966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:50.662636736 +0000 UTC m=+611.172797693" watchObservedRunningTime="2026-07-06 23:58:50.663683137 +0000 UTC m=+611.173844093" Jul 06 23:58:59.011161 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:59.011119 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:58:59.325749 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:59.325669 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:59.325749 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:59.325736 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:59.338242 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:59.338211 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:58:59.686482 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:58:59.686448 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnrtq" Jul 06 23:59:04.204003 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:04.203915 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:59:13.883168 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.883128 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl"] Jul 06 23:59:13.885797 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.885767 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:13.888534 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.888509 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 06 23:59:13.897686 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.897660 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl"] Jul 06 23:59:13.915410 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.915374 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:59:13.969770 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.969729 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:13.969981 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.969780 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4978afe4-69a4-42c3-afba-3d7b99d6182c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:13.969981 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.969901 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cx5r\" (UniqueName: \"kubernetes.io/projected/4978afe4-69a4-42c3-afba-3d7b99d6182c-kube-api-access-6cx5r\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:13.969981 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.969924 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:13.969981 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.969948 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:13.969981 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:13.969975 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.071324 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.071286 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.071537 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.071337 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4978afe4-69a4-42c3-afba-3d7b99d6182c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.071537 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.071390 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cx5r\" (UniqueName: \"kubernetes.io/projected/4978afe4-69a4-42c3-afba-3d7b99d6182c-kube-api-access-6cx5r\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.071537 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.071413 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.071537 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.071448 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.071537 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.071478 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.071903 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.071788 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.072435 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.072395 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.072537 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.072492 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.074755 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.074729 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4978afe4-69a4-42c3-afba-3d7b99d6182c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.079045 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.078817 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4978afe4-69a4-42c3-afba-3d7b99d6182c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.080095 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.080071 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cx5r\" (UniqueName: \"kubernetes.io/projected/4978afe4-69a4-42c3-afba-3d7b99d6182c-kube-api-access-6cx5r\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl\" (UID: \"4978afe4-69a4-42c3-afba-3d7b99d6182c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.195721 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.195680 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:14.335222 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.335198 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl"] Jul 06 23:59:14.337669 ip-10-0-130-213 kubenswrapper[2558]: W0706 23:59:14.337637 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4978afe4_69a4_42c3_afba_3d7b99d6182c.slice/crio-f51bfe198e8713560c740504af9f77f320fdf8326cbc0314ceab2fb90ee19265 WatchSource:0}: Error finding container f51bfe198e8713560c740504af9f77f320fdf8326cbc0314ceab2fb90ee19265: Status 404 returned error can't find the container with id f51bfe198e8713560c740504af9f77f320fdf8326cbc0314ceab2fb90ee19265 Jul 06 23:59:14.741301 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.741207 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" event={"ID":"4978afe4-69a4-42c3-afba-3d7b99d6182c","Type":"ContainerStarted","Data":"4f9cb71deb4ed41b6eb84fb045706926feb4a1c75170aebc57ee995701d545cc"} Jul 06 23:59:14.741301 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.741253 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" event={"ID":"4978afe4-69a4-42c3-afba-3d7b99d6182c","Type":"ContainerStarted","Data":"f51bfe198e8713560c740504af9f77f320fdf8326cbc0314ceab2fb90ee19265"} Jul 06 23:59:14.760060 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:14.759997 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" podStartSLOduration=1.6147696809999998 podStartE2EDuration="1.759973921s" totalImagesPullingTime="145.20424ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:59:14.759110264 +0000 UTC m=+635.269271238" watchObservedRunningTime="2026-07-06 23:59:14.759973921 +0000 UTC m=+635.270134879" Jul 06 23:59:24.196775 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:24.196738 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:24.197227 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:24.196803 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:24.209251 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:24.209226 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:24.797298 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:24.797267 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl" Jul 06 23:59:59.167035 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:59.166978 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 06 23:59:59.264612 ip-10-0-130-213 kubenswrapper[2558]: I0706 23:59:59.264564 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:00:00.370580 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:00:00.370541 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:00:00.469175 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:00:00.469138 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:00:40.772328 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:00:40.772287 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:04.123102 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:04.123060 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:10.808537 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:10.808503 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:10.908754 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:10.908717 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d86fcfb77-mxz8j"] Jul 07 00:01:10.908970 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:10.908942 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" podUID="7cd2be0a-44e9-4f87-9694-8da28b325194" containerName="manager" containerID="cri-o://3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab" gracePeriod=30 Jul 07 00:01:11.165034 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.165007 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 07 00:01:11.246666 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.246624 2558 generic.go:356] "Generic (PLEG): container finished" podID="7cd2be0a-44e9-4f87-9694-8da28b325194" containerID="3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab" exitCode=0 Jul 07 00:01:11.246666 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.246669 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" Jul 07 00:01:11.246891 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.246704 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" event={"ID":"7cd2be0a-44e9-4f87-9694-8da28b325194","Type":"ContainerDied","Data":"3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab"} Jul 07 00:01:11.246891 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.246760 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d86fcfb77-mxz8j" event={"ID":"7cd2be0a-44e9-4f87-9694-8da28b325194","Type":"ContainerDied","Data":"67a85ad0bfc1887d3a8a2f5e9247b06234fc681dad3e0f9ea76095974da9d5b1"} Jul 07 00:01:11.246891 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.246784 2558 scope.go:117] "RemoveContainer" containerID="3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab" Jul 07 00:01:11.255377 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.255357 2558 scope.go:117] "RemoveContainer" containerID="3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab" Jul 07 00:01:11.255663 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:01:11.255641 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab\": container with ID starting with 3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab not found: ID does not exist" containerID="3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab" Jul 07 00:01:11.255712 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.255672 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab"} err="failed to get container status \"3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab\": rpc error: code = NotFound desc = could not find container \"3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab\": container with ID starting with 3c61b4b7f73a9b76d361b5bed9749a7a48e29922e05c6d2cd6570393ff6a18ab not found: ID does not exist" Jul 07 00:01:11.325318 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.325207 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld7hm\" (UniqueName: \"kubernetes.io/projected/7cd2be0a-44e9-4f87-9694-8da28b325194-kube-api-access-ld7hm\") pod \"7cd2be0a-44e9-4f87-9694-8da28b325194\" (UID: \"7cd2be0a-44e9-4f87-9694-8da28b325194\") " Jul 07 00:01:11.325318 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.325257 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cd2be0a-44e9-4f87-9694-8da28b325194-webhook-cert\") pod \"7cd2be0a-44e9-4f87-9694-8da28b325194\" (UID: \"7cd2be0a-44e9-4f87-9694-8da28b325194\") " Jul 07 00:01:11.327502 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.327468 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cd2be0a-44e9-4f87-9694-8da28b325194-kube-api-access-ld7hm" (OuterVolumeSpecName: "kube-api-access-ld7hm") pod "7cd2be0a-44e9-4f87-9694-8da28b325194" (UID: "7cd2be0a-44e9-4f87-9694-8da28b325194"). InnerVolumeSpecName "kube-api-access-ld7hm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:01:11.327644 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.327510 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cd2be0a-44e9-4f87-9694-8da28b325194-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7cd2be0a-44e9-4f87-9694-8da28b325194" (UID: "7cd2be0a-44e9-4f87-9694-8da28b325194"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:01:11.426266 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.426226 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld7hm\" (UniqueName: \"kubernetes.io/projected/7cd2be0a-44e9-4f87-9694-8da28b325194-kube-api-access-ld7hm\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:01:11.426266 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.426262 2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cd2be0a-44e9-4f87-9694-8da28b325194-webhook-cert\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:01:11.569387 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.569343 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d86fcfb77-mxz8j"] Jul 07 00:01:11.570867 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:11.570830 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d86fcfb77-mxz8j"] Jul 07 00:01:12.051869 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.051819 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cd2be0a-44e9-4f87-9694-8da28b325194" path="/var/lib/kubelet/pods/7cd2be0a-44e9-4f87-9694-8da28b325194/volumes" Jul 07 00:01:12.247641 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.247600 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:12.346037 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.345959 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d86fcfb77-62mlh"] Jul 07 00:01:12.346297 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.346285 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cd2be0a-44e9-4f87-9694-8da28b325194" containerName="manager" Jul 07 00:01:12.346297 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.346298 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cd2be0a-44e9-4f87-9694-8da28b325194" containerName="manager" Jul 07 00:01:12.346400 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.346351 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cd2be0a-44e9-4f87-9694-8da28b325194" containerName="manager" Jul 07 00:01:12.350585 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.350564 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d86fcfb77-62mlh" Jul 07 00:01:12.354585 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.354558 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 00:01:12.354585 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.354579 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nw684\"" Jul 07 00:01:12.354760 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.354579 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 00:01:12.360968 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.360923 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d86fcfb77-62mlh"] Jul 07 00:01:12.436237 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.436202 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c4pv\" (UniqueName: \"kubernetes.io/projected/143497c2-c2b8-4a7c-8bd4-8b6645368ff6-kube-api-access-4c4pv\") pod \"maas-controller-6d86fcfb77-62mlh\" (UID: \"143497c2-c2b8-4a7c-8bd4-8b6645368ff6\") " pod="opendatahub/maas-controller-6d86fcfb77-62mlh" Jul 07 00:01:12.436420 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.436283 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/143497c2-c2b8-4a7c-8bd4-8b6645368ff6-webhook-cert\") pod \"maas-controller-6d86fcfb77-62mlh\" (UID: \"143497c2-c2b8-4a7c-8bd4-8b6645368ff6\") " pod="opendatahub/maas-controller-6d86fcfb77-62mlh" Jul 07 00:01:12.536931 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.536845 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c4pv\" (UniqueName: \"kubernetes.io/projected/143497c2-c2b8-4a7c-8bd4-8b6645368ff6-kube-api-access-4c4pv\") pod \"maas-controller-6d86fcfb77-62mlh\" (UID: \"143497c2-c2b8-4a7c-8bd4-8b6645368ff6\") " pod="opendatahub/maas-controller-6d86fcfb77-62mlh" Jul 07 00:01:12.537083 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.537005 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/143497c2-c2b8-4a7c-8bd4-8b6645368ff6-webhook-cert\") pod \"maas-controller-6d86fcfb77-62mlh\" (UID: \"143497c2-c2b8-4a7c-8bd4-8b6645368ff6\") " pod="opendatahub/maas-controller-6d86fcfb77-62mlh" Jul 07 00:01:12.539699 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.539665 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/143497c2-c2b8-4a7c-8bd4-8b6645368ff6-webhook-cert\") pod \"maas-controller-6d86fcfb77-62mlh\" (UID: \"143497c2-c2b8-4a7c-8bd4-8b6645368ff6\") " pod="opendatahub/maas-controller-6d86fcfb77-62mlh" Jul 07 00:01:12.546584 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.546557 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c4pv\" (UniqueName: \"kubernetes.io/projected/143497c2-c2b8-4a7c-8bd4-8b6645368ff6-kube-api-access-4c4pv\") pod \"maas-controller-6d86fcfb77-62mlh\" (UID: \"143497c2-c2b8-4a7c-8bd4-8b6645368ff6\") " pod="opendatahub/maas-controller-6d86fcfb77-62mlh" Jul 07 00:01:12.661769 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.661671 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d86fcfb77-62mlh" Jul 07 00:01:12.798019 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:12.797994 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d86fcfb77-62mlh"] Jul 07 00:01:12.800186 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:01:12.800156 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod143497c2_c2b8_4a7c_8bd4_8b6645368ff6.slice/crio-ad3b7bad4752735af34fdb2bfddfdafa6d59f7e245cbd347101574e7293390d9 WatchSource:0}: Error finding container ad3b7bad4752735af34fdb2bfddfdafa6d59f7e245cbd347101574e7293390d9: Status 404 returned error can't find the container with id ad3b7bad4752735af34fdb2bfddfdafa6d59f7e245cbd347101574e7293390d9 Jul 07 00:01:13.256822 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:13.256783 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d86fcfb77-62mlh" event={"ID":"143497c2-c2b8-4a7c-8bd4-8b6645368ff6","Type":"ContainerStarted","Data":"ad3b7bad4752735af34fdb2bfddfdafa6d59f7e245cbd347101574e7293390d9"} Jul 07 00:01:14.262818 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:14.262784 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d86fcfb77-62mlh" event={"ID":"143497c2-c2b8-4a7c-8bd4-8b6645368ff6","Type":"ContainerStarted","Data":"3cc9a64a1667e38fd976d85434895cf749cef08591cf27d8eba3afe78e841f3d"} Jul 07 00:01:14.263224 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:14.262921 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d86fcfb77-62mlh" Jul 07 00:01:14.286415 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:14.286364 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d86fcfb77-62mlh" podStartSLOduration=1.8015652580000001 podStartE2EDuration="2.286346332s" totalImagesPullingTime="484.781074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:14.284490384 +0000 UTC m=+754.794651355" watchObservedRunningTime="2026-07-07 00:01:14.286346332 +0000 UTC m=+754.796507289" Jul 07 00:01:22.585148 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:22.585107 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:22.642701 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:22.642664 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:25.272786 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:25.272752 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d86fcfb77-62mlh" Jul 07 00:01:53.653949 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:53.653912 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:53.749725 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:53.749688 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:53.849871 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:53.849817 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:54.558407 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:54.558358 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:54.662133 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:54.662091 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:59.426069 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:59.426028 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:59.529878 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:59.529815 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:59.625034 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:59.624992 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:01:59.780507 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:01:59.780418 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:02:05.529053 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:02:05.528965 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:02:05.593568 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:02:05.593524 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:02:53.355264 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:02:53.355218 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:03:39.996969 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:03:39.996881 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:03:40.000055 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:03:40.000035 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:04:10.593149 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:04:10.593113 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:05:07.912451 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:05:07.912368 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:07.400078 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:08:07.399987 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:40.023679 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:08:40.023641 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:08:40.026117 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:08:40.024137 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:09:15.709951 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:09:15.709912 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:17.791256 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:09:17.791225 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:29.007454 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:11:29.007360 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:31.243524 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:31.243432 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:36.629036 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:36.629003 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:44.886625 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:44.886586 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:49.995592 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:49.995560 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:52.848081 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:52.848037 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:53.008332 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:53.008292 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:53.320994 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:53.320955 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:59.118275 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.118239 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:59.219248 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.219211 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:59.310276 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.310243 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh"] Jul 07 00:12:59.313560 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.313540 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.316720 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.316690 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e5d82bf8-openshift-default-dockercfg-xjw4s\"" Jul 07 00:12:59.336910 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.336880 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh"] Jul 07 00:12:59.340391 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.340365 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:59.401405 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.401304 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:59.438298 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.438254 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b2ef34e-b078-42b3-b0a3-27829f731948-istiod-ca-cert\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.438298 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.438300 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-credential-socket\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.438511 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.438322 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-envoy\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.438511 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.438349 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kck5\" (UniqueName: \"kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-kube-api-access-4kck5\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.438511 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.438417 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-podinfo\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.438511 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.438449 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-certs\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.438511 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.438474 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-socket\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.438511 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.438490 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-data\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.438694 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.438522 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-token\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.539766 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.539724 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-podinfo\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540006 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.539778 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-certs\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540006 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.539816 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-socket\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540006 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.539841 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-data\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540006 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.539918 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-token\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540006 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.539957 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b2ef34e-b078-42b3-b0a3-27829f731948-istiod-ca-cert\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540006 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.539986 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-credential-socket\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540312 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.540013 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-envoy\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540312 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.540035 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kck5\" (UniqueName: \"kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-kube-api-access-4kck5\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540312 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.540269 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-certs\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540470 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.540311 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-socket\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540470 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.540376 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-data\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540582 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.540550 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-credential-socket\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.540673 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.540652 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b2ef34e-b078-42b3-b0a3-27829f731948-istiod-ca-cert\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.542394 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.542372 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-envoy\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.542503 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.542411 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-podinfo\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.549062 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.549039 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-token\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.549203 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.549185 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kck5\" (UniqueName: \"kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-kube-api-access-4kck5\") pod \"e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.627058 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.627019 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:12:59.772733 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.772704 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh"] Jul 07 00:12:59.775336 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:12:59.775298 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b2ef34e_b078_42b3_b0a3_27829f731948.slice/crio-9aecc62b25ac1c13278544c50e8b2d87480314af42a67deeb7f59136e32f9dcf WatchSource:0}: Error finding container 9aecc62b25ac1c13278544c50e8b2d87480314af42a67deeb7f59136e32f9dcf: Status 404 returned error can't find the container with id 9aecc62b25ac1c13278544c50e8b2d87480314af42a67deeb7f59136e32f9dcf Jul 07 00:12:59.777535 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.777504 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:12:59.777608 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.777575 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:12:59.777608 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:12:59.777605 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:13:00.375125 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.375086 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" event={"ID":"6b2ef34e-b078-42b3-b0a3-27829f731948","Type":"ContainerStarted","Data":"f39b579603ae3ffa5c92b611db57aca0e5ddcebae84838fd2381cc13233e841d"} Jul 07 00:13:00.375530 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.375149 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" event={"ID":"6b2ef34e-b078-42b3-b0a3-27829f731948","Type":"ContainerStarted","Data":"9aecc62b25ac1c13278544c50e8b2d87480314af42a67deeb7f59136e32f9dcf"} Jul 07 00:13:00.395091 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.395029 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" podStartSLOduration=1.394991027 podStartE2EDuration="1.394991027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:13:00.393630323 +0000 UTC m=+1460.903791317" watchObservedRunningTime="2026-07-07 00:13:00.394991027 +0000 UTC m=+1460.905151985" Jul 07 00:13:00.458814 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.458753 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:00.529789 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.529719 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn"] Jul 07 00:13:00.533399 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.533373 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.536388 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.536363 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-e5d82bf8-serving-cert\"" Jul 07 00:13:00.545894 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.545830 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn"] Jul 07 00:13:00.561519 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.561484 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:00.627697 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.627594 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:13:00.651427 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.651398 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:13:00.651592 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.651541 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3356adbb-944d-47a6-920d-f6c54cf9b814-openshift-service-ca\") pod \"maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.651684 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.651656 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5cht\" (UniqueName: \"kubernetes.io/projected/3356adbb-944d-47a6-920d-f6c54cf9b814-kube-api-access-f5cht\") pod \"maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.651778 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.651760 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3356adbb-944d-47a6-920d-f6c54cf9b814-maas-api-tls\") pod \"maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.655452 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.655415 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:00.753123 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.753066 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3356adbb-944d-47a6-920d-f6c54cf9b814-openshift-service-ca\") pod \"maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.753123 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.753127 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5cht\" (UniqueName: \"kubernetes.io/projected/3356adbb-944d-47a6-920d-f6c54cf9b814-kube-api-access-f5cht\") pod \"maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.753387 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.753175 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3356adbb-944d-47a6-920d-f6c54cf9b814-maas-api-tls\") pod \"maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.753972 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.753938 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3356adbb-944d-47a6-920d-f6c54cf9b814-openshift-service-ca\") pod \"maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.754096 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.754069 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:00.755917 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.755895 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3356adbb-944d-47a6-920d-f6c54cf9b814-maas-api-tls\") pod \"maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.763217 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.763194 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5cht\" (UniqueName: \"kubernetes.io/projected/3356adbb-944d-47a6-920d-f6c54cf9b814-kube-api-access-f5cht\") pod \"maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.845355 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.845313 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:00.986908 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.986881 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn"] Jul 07 00:13:00.989666 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:13:00.989638 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3356adbb_944d_47a6_920d_f6c54cf9b814.slice/crio-a0fdbfa2b0a6e4775fd518303d00cc7b79d5c18e346ec7d6f72c07ccf97b0b40 WatchSource:0}: Error finding container a0fdbfa2b0a6e4775fd518303d00cc7b79d5c18e346ec7d6f72c07ccf97b0b40: Status 404 returned error can't find the container with id a0fdbfa2b0a6e4775fd518303d00cc7b79d5c18e346ec7d6f72c07ccf97b0b40 Jul 07 00:13:00.991094 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:00.991076 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:13:01.015843 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:01.015810 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:01.113081 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:01.113044 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:01.380683 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:01.380645 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" event={"ID":"3356adbb-944d-47a6-920d-f6c54cf9b814","Type":"ContainerStarted","Data":"a0fdbfa2b0a6e4775fd518303d00cc7b79d5c18e346ec7d6f72c07ccf97b0b40"} Jul 07 00:13:01.381168 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:01.380965 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:13:01.381809 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:01.381792 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:13:02.386094 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:02.386054 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" event={"ID":"3356adbb-944d-47a6-920d-f6c54cf9b814","Type":"ContainerStarted","Data":"65f67459c25ac17ab589fe5dcdb8ff9a134c7cbc8d19e4eaed3baa347bf6b3e7"} Jul 07 00:13:02.404349 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:02.404292 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" podStartSLOduration=1.843484547 podStartE2EDuration="2.404275043s" totalImagesPullingTime="560.790496ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:13:02.40250729 +0000 UTC m=+1462.912668308" watchObservedRunningTime="2026-07-07 00:13:02.404275043 +0000 UTC m=+1462.914436000" Jul 07 00:13:03.110644 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:03.110614 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:03.391332 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:03.391249 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:05.655356 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:05.655314 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:05.947414 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:05.947379 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh"] Jul 07 00:13:05.947611 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:05.947584 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" podUID="6b2ef34e-b078-42b3-b0a3-27829f731948" containerName="istio-proxy" containerID="cri-o://f39b579603ae3ffa5c92b611db57aca0e5ddcebae84838fd2381cc13233e841d" gracePeriod=30 Jul 07 00:13:05.970921 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:05.970885 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:06.071150 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.071108 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:06.219940 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.219835 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn"] Jul 07 00:13:06.220103 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.220060 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" podUID="3356adbb-944d-47a6-920d-f6c54cf9b814" containerName="maas-api" containerID="cri-o://65f67459c25ac17ab589fe5dcdb8ff9a134c7cbc8d19e4eaed3baa347bf6b3e7" gracePeriod=30 Jul 07 00:13:06.225575 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.225543 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:06.250127 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.250100 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:06.423185 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.422555 2558 generic.go:356] "Generic (PLEG): container finished" podID="3356adbb-944d-47a6-920d-f6c54cf9b814" containerID="65f67459c25ac17ab589fe5dcdb8ff9a134c7cbc8d19e4eaed3baa347bf6b3e7" exitCode=0 Jul 07 00:13:06.423185 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.422691 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" event={"ID":"3356adbb-944d-47a6-920d-f6c54cf9b814","Type":"ContainerDied","Data":"65f67459c25ac17ab589fe5dcdb8ff9a134c7cbc8d19e4eaed3baa347bf6b3e7"} Jul 07 00:13:06.490864 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.490441 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:06.608316 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.608278 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5cht\" (UniqueName: \"kubernetes.io/projected/3356adbb-944d-47a6-920d-f6c54cf9b814-kube-api-access-f5cht\") pod \"3356adbb-944d-47a6-920d-f6c54cf9b814\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " Jul 07 00:13:06.608508 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.608365 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3356adbb-944d-47a6-920d-f6c54cf9b814-openshift-service-ca\") pod \"3356adbb-944d-47a6-920d-f6c54cf9b814\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " Jul 07 00:13:06.608571 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.608552 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3356adbb-944d-47a6-920d-f6c54cf9b814-maas-api-tls\") pod \"3356adbb-944d-47a6-920d-f6c54cf9b814\" (UID: \"3356adbb-944d-47a6-920d-f6c54cf9b814\") " Jul 07 00:13:06.608660 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.608636 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3356adbb-944d-47a6-920d-f6c54cf9b814-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3356adbb-944d-47a6-920d-f6c54cf9b814" (UID: "3356adbb-944d-47a6-920d-f6c54cf9b814"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:13:06.608917 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.608898 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3356adbb-944d-47a6-920d-f6c54cf9b814-openshift-service-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:06.610595 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.610568 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3356adbb-944d-47a6-920d-f6c54cf9b814-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3356adbb-944d-47a6-920d-f6c54cf9b814" (UID: "3356adbb-944d-47a6-920d-f6c54cf9b814"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:13:06.610683 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.610647 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3356adbb-944d-47a6-920d-f6c54cf9b814-kube-api-access-f5cht" (OuterVolumeSpecName: "kube-api-access-f5cht") pod "3356adbb-944d-47a6-920d-f6c54cf9b814" (UID: "3356adbb-944d-47a6-920d-f6c54cf9b814"). InnerVolumeSpecName "kube-api-access-f5cht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:13:06.710176 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.710124 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3356adbb-944d-47a6-920d-f6c54cf9b814-maas-api-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:06.710176 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:06.710170 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5cht\" (UniqueName: \"kubernetes.io/projected/3356adbb-944d-47a6-920d-f6c54cf9b814-kube-api-access-f5cht\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:07.429146 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:07.429115 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" Jul 07 00:13:07.429320 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:07.429147 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn" event={"ID":"3356adbb-944d-47a6-920d-f6c54cf9b814","Type":"ContainerDied","Data":"a0fdbfa2b0a6e4775fd518303d00cc7b79d5c18e346ec7d6f72c07ccf97b0b40"} Jul 07 00:13:07.429320 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:07.429210 2558 scope.go:117] "RemoveContainer" containerID="65f67459c25ac17ab589fe5dcdb8ff9a134c7cbc8d19e4eaed3baa347bf6b3e7" Jul 07 00:13:07.450303 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:07.450276 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn"] Jul 07 00:13:07.454201 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:07.454172 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-e5d82bf8-69d6ccb7cc-tznvn"] Jul 07 00:13:07.895876 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:07.890751 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:08.051631 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:08.051599 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3356adbb-944d-47a6-920d-f6c54cf9b814" path="/var/lib/kubelet/pods/3356adbb-944d-47a6-920d-f6c54cf9b814/volumes" Jul 07 00:13:11.450931 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.450842 2558 generic.go:356] "Generic (PLEG): container finished" podID="6b2ef34e-b078-42b3-b0a3-27829f731948" containerID="f39b579603ae3ffa5c92b611db57aca0e5ddcebae84838fd2381cc13233e841d" exitCode=0 Jul 07 00:13:11.450931 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.450885 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" event={"ID":"6b2ef34e-b078-42b3-b0a3-27829f731948","Type":"ContainerDied","Data":"f39b579603ae3ffa5c92b611db57aca0e5ddcebae84838fd2381cc13233e841d"} Jul 07 00:13:11.489968 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.489943 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:13:11.655562 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.655463 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-envoy\") pod \"6b2ef34e-b078-42b3-b0a3-27829f731948\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " Jul 07 00:13:11.655562 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.655505 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-token\") pod \"6b2ef34e-b078-42b3-b0a3-27829f731948\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " Jul 07 00:13:11.655562 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.655535 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b2ef34e-b078-42b3-b0a3-27829f731948-istiod-ca-cert\") pod \"6b2ef34e-b078-42b3-b0a3-27829f731948\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " Jul 07 00:13:11.655562 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.655564 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kck5\" (UniqueName: \"kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-kube-api-access-4kck5\") pod \"6b2ef34e-b078-42b3-b0a3-27829f731948\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " Jul 07 00:13:11.655949 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.655752 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-podinfo\") pod \"6b2ef34e-b078-42b3-b0a3-27829f731948\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " Jul 07 00:13:11.655949 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.655836 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-certs\") pod \"6b2ef34e-b078-42b3-b0a3-27829f731948\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " Jul 07 00:13:11.655949 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.655904 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-socket\") pod \"6b2ef34e-b078-42b3-b0a3-27829f731948\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " Jul 07 00:13:11.655949 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.655932 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-credential-socket\") pod \"6b2ef34e-b078-42b3-b0a3-27829f731948\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " Jul 07 00:13:11.656165 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.655959 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-data\") pod \"6b2ef34e-b078-42b3-b0a3-27829f731948\" (UID: \"6b2ef34e-b078-42b3-b0a3-27829f731948\") " Jul 07 00:13:11.656165 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.656093 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6b2ef34e-b078-42b3-b0a3-27829f731948" (UID: "6b2ef34e-b078-42b3-b0a3-27829f731948"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:13:11.656165 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.656115 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b2ef34e-b078-42b3-b0a3-27829f731948-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6b2ef34e-b078-42b3-b0a3-27829f731948" (UID: "6b2ef34e-b078-42b3-b0a3-27829f731948"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:13:11.656330 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.656308 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6b2ef34e-b078-42b3-b0a3-27829f731948" (UID: "6b2ef34e-b078-42b3-b0a3-27829f731948"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:13:11.656507 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.656460 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6b2ef34e-b078-42b3-b0a3-27829f731948" (UID: "6b2ef34e-b078-42b3-b0a3-27829f731948"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:13:11.656507 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.656487 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-certs\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:11.656507 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.656509 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-workload-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:11.656715 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.656523 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b2ef34e-b078-42b3-b0a3-27829f731948-istiod-ca-cert\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:11.656933 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.656903 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-data" (OuterVolumeSpecName: "istio-data") pod "6b2ef34e-b078-42b3-b0a3-27829f731948" (UID: "6b2ef34e-b078-42b3-b0a3-27829f731948"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:13:11.658098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.658079 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-kube-api-access-4kck5" (OuterVolumeSpecName: "kube-api-access-4kck5") pod "6b2ef34e-b078-42b3-b0a3-27829f731948" (UID: "6b2ef34e-b078-42b3-b0a3-27829f731948"). InnerVolumeSpecName "kube-api-access-4kck5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:13:11.658191 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.658107 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-token" (OuterVolumeSpecName: "istio-token") pod "6b2ef34e-b078-42b3-b0a3-27829f731948" (UID: "6b2ef34e-b078-42b3-b0a3-27829f731948"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:13:11.658322 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.658303 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6b2ef34e-b078-42b3-b0a3-27829f731948" (UID: "6b2ef34e-b078-42b3-b0a3-27829f731948"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:13:11.658372 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.658330 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6b2ef34e-b078-42b3-b0a3-27829f731948" (UID: "6b2ef34e-b078-42b3-b0a3-27829f731948"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:13:11.757496 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.757456 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-credential-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:11.757496 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.757488 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-data\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:11.757496 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.757498 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-envoy\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:11.757496 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.757506 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-token\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:11.757759 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.757538 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kck5\" (UniqueName: \"kubernetes.io/projected/6b2ef34e-b078-42b3-b0a3-27829f731948-kube-api-access-4kck5\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:11.757759 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.757548 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b2ef34e-b078-42b3-b0a3-27829f731948-istio-podinfo\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:11.857524 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.857485 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:11.957634 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:11.957600 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:12.040873 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.040820 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf"] Jul 07 00:13:12.041212 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.041196 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b2ef34e-b078-42b3-b0a3-27829f731948" containerName="istio-proxy" Jul 07 00:13:12.041275 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.041214 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b2ef34e-b078-42b3-b0a3-27829f731948" containerName="istio-proxy" Jul 07 00:13:12.041275 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.041224 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3356adbb-944d-47a6-920d-f6c54cf9b814" containerName="maas-api" Jul 07 00:13:12.041275 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.041229 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3356adbb-944d-47a6-920d-f6c54cf9b814" containerName="maas-api" Jul 07 00:13:12.041425 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.041296 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3356adbb-944d-47a6-920d-f6c54cf9b814" containerName="maas-api" Jul 07 00:13:12.041425 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.041307 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b2ef34e-b078-42b3-b0a3-27829f731948" containerName="istio-proxy" Jul 07 00:13:12.043936 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.043894 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.047562 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.047533 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8cd6b865-openshift-default-dockercfg-b8ft9\"" Jul 07 00:13:12.062073 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.062046 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf"] Jul 07 00:13:12.085309 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.085279 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:12.156414 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.156380 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:12.163320 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.163284 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-data\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.163320 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.163320 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl7vk\" (UniqueName: \"kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-kube-api-access-dl7vk\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.163743 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.163391 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-socket\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.163743 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.163413 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-credential-socket\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.163743 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.163444 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-istio-token\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.163743 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.163461 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90564228-4e90-4549-ab63-e7e45c2687d2-istio-podinfo\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.163743 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.163512 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-certs\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.163743 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.163547 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90564228-4e90-4549-ab63-e7e45c2687d2-istiod-ca-cert\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.163743 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.163581 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-envoy\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.264539 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.264448 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-socket\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.264539 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.264485 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-credential-socket\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.264539 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.264514 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-istio-token\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.264801 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.264565 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90564228-4e90-4549-ab63-e7e45c2687d2-istio-podinfo\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.264801 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.264629 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-certs\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.264801 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.264657 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90564228-4e90-4549-ab63-e7e45c2687d2-istiod-ca-cert\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.264801 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.264703 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-envoy\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.264801 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.264749 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-data\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.264801 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.264777 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl7vk\" (UniqueName: \"kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-kube-api-access-dl7vk\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.265138 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.264997 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-credential-socket\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.265138 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.265049 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-socket\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.265249 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.265147 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-certs\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.265306 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.265269 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-data\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.265516 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.265494 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90564228-4e90-4549-ab63-e7e45c2687d2-istiod-ca-cert\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.267023 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.267003 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-envoy\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.267147 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.267131 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90564228-4e90-4549-ab63-e7e45c2687d2-istio-podinfo\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.272784 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.272761 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-istio-token\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.273352 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.273332 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl7vk\" (UniqueName: \"kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-kube-api-access-dl7vk\") pod \"e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.356730 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.356684 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:12.457201 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.457160 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" event={"ID":"6b2ef34e-b078-42b3-b0a3-27829f731948","Type":"ContainerDied","Data":"9aecc62b25ac1c13278544c50e8b2d87480314af42a67deeb7f59136e32f9dcf"} Jul 07 00:13:12.457650 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.457252 2558 scope.go:117] "RemoveContainer" containerID="f39b579603ae3ffa5c92b611db57aca0e5ddcebae84838fd2381cc13233e841d" Jul 07 00:13:12.457650 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.457180 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh" Jul 07 00:13:12.478631 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.478604 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh"] Jul 07 00:13:12.482323 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.482292 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e5d82bf8-openshift-default-5f5c66c9ff-mbkjh"] Jul 07 00:13:12.501991 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.501965 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf"] Jul 07 00:13:12.504081 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:13:12.504056 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90564228_4e90_4549_ab63_e7e45c2687d2.slice/crio-8bfbb6829487f110364fd392d004603b6a7ec5658d0ffa8d9ec5f9d61404a4f8 WatchSource:0}: Error finding container 8bfbb6829487f110364fd392d004603b6a7ec5658d0ffa8d9ec5f9d61404a4f8: Status 404 returned error can't find the container with id 8bfbb6829487f110364fd392d004603b6a7ec5658d0ffa8d9ec5f9d61404a4f8 Jul 07 00:13:12.506084 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.506048 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:13:12.506177 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.506120 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:13:12.506177 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:12.506147 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:13:13.200226 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.200187 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:13.264557 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.264481 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs"] Jul 07 00:13:13.267294 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.267267 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.270023 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.270005 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-8cd6b865-serving-cert\"" Jul 07 00:13:13.280274 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.280243 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs"] Jul 07 00:13:13.307906 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.307864 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:13.375200 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.375154 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzj22\" (UniqueName: \"kubernetes.io/projected/605bd53c-d414-4a19-ae73-7afd50d85e18-kube-api-access-vzj22\") pod \"maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.375386 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.375210 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/605bd53c-d414-4a19-ae73-7afd50d85e18-openshift-service-ca\") pod \"maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.375386 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.375312 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/605bd53c-d414-4a19-ae73-7afd50d85e18-maas-api-tls\") pod \"maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.453564 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.453473 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:13.465635 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.465584 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" event={"ID":"90564228-4e90-4549-ab63-e7e45c2687d2","Type":"ContainerStarted","Data":"6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8"} Jul 07 00:13:13.466160 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.465657 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" event={"ID":"90564228-4e90-4549-ab63-e7e45c2687d2","Type":"ContainerStarted","Data":"8bfbb6829487f110364fd392d004603b6a7ec5658d0ffa8d9ec5f9d61404a4f8"} Jul 07 00:13:13.475951 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.475915 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/605bd53c-d414-4a19-ae73-7afd50d85e18-maas-api-tls\") pod \"maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.476148 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.476035 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzj22\" (UniqueName: \"kubernetes.io/projected/605bd53c-d414-4a19-ae73-7afd50d85e18-kube-api-access-vzj22\") pod \"maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.476148 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.476082 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/605bd53c-d414-4a19-ae73-7afd50d85e18-openshift-service-ca\") pod \"maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.476786 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.476765 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/605bd53c-d414-4a19-ae73-7afd50d85e18-openshift-service-ca\") pod \"maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.478640 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.478610 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/605bd53c-d414-4a19-ae73-7afd50d85e18-maas-api-tls\") pod \"maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.497137 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.497110 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzj22\" (UniqueName: \"kubernetes.io/projected/605bd53c-d414-4a19-ae73-7afd50d85e18-kube-api-access-vzj22\") pod \"maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.514883 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.514809 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" podStartSLOduration=1.514792011 podStartE2EDuration="1.514792011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:13:13.513374261 +0000 UTC m=+1474.023535232" watchObservedRunningTime="2026-07-07 00:13:13.514792011 +0000 UTC m=+1474.024952972" Jul 07 00:13:13.562506 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.562463 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:13.578941 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.578908 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:13.721664 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.721628 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs"] Jul 07 00:13:13.723656 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:13:13.723619 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod605bd53c_d414_4a19_ae73_7afd50d85e18.slice/crio-9977646b2a1c34e370b8508c7e5d23c6968b2b383fd8917b166c2586ddc0ba21 WatchSource:0}: Error finding container 9977646b2a1c34e370b8508c7e5d23c6968b2b383fd8917b166c2586ddc0ba21: Status 404 returned error can't find the container with id 9977646b2a1c34e370b8508c7e5d23c6968b2b383fd8917b166c2586ddc0ba21 Jul 07 00:13:13.861935 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.861888 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:13.962502 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:13.962468 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:14.051741 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:14.051658 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b2ef34e-b078-42b3-b0a3-27829f731948" path="/var/lib/kubelet/pods/6b2ef34e-b078-42b3-b0a3-27829f731948/volumes" Jul 07 00:13:14.357811 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:14.357722 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:14.363266 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:14.363238 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:14.472667 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:14.472632 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" event={"ID":"605bd53c-d414-4a19-ae73-7afd50d85e18","Type":"ContainerStarted","Data":"81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39"} Jul 07 00:13:14.473173 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:14.472721 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" event={"ID":"605bd53c-d414-4a19-ae73-7afd50d85e18","Type":"ContainerStarted","Data":"9977646b2a1c34e370b8508c7e5d23c6968b2b383fd8917b166c2586ddc0ba21"} Jul 07 00:13:14.473173 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:14.472980 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:14.473173 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:14.473032 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:14.474063 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:14.474043 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:14.493911 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:14.493867 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" podStartSLOduration=1.09916845 podStartE2EDuration="1.49383679s" totalImagesPullingTime="394.66834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:13:14.493221703 +0000 UTC m=+1475.003382662" watchObservedRunningTime="2026-07-07 00:13:14.49383679 +0000 UTC m=+1475.003997746" Jul 07 00:13:15.539127 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:15.539084 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:18.436892 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:18.436829 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:18.765911 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:18.765794 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs"] Jul 07 00:13:18.766638 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:18.766080 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" podUID="605bd53c-d414-4a19-ae73-7afd50d85e18" containerName="maas-api" containerID="cri-o://81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39" gracePeriod=30 Jul 07 00:13:18.787931 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:18.787892 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:19.053533 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.053508 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:19.129294 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.129261 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/605bd53c-d414-4a19-ae73-7afd50d85e18-openshift-service-ca\") pod \"605bd53c-d414-4a19-ae73-7afd50d85e18\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " Jul 07 00:13:19.129484 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.129325 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzj22\" (UniqueName: \"kubernetes.io/projected/605bd53c-d414-4a19-ae73-7afd50d85e18-kube-api-access-vzj22\") pod \"605bd53c-d414-4a19-ae73-7afd50d85e18\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " Jul 07 00:13:19.129484 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.129374 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/605bd53c-d414-4a19-ae73-7afd50d85e18-maas-api-tls\") pod \"605bd53c-d414-4a19-ae73-7afd50d85e18\" (UID: \"605bd53c-d414-4a19-ae73-7afd50d85e18\") " Jul 07 00:13:19.129661 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.129635 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/605bd53c-d414-4a19-ae73-7afd50d85e18-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "605bd53c-d414-4a19-ae73-7afd50d85e18" (UID: "605bd53c-d414-4a19-ae73-7afd50d85e18"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:13:19.131699 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.131670 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/605bd53c-d414-4a19-ae73-7afd50d85e18-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "605bd53c-d414-4a19-ae73-7afd50d85e18" (UID: "605bd53c-d414-4a19-ae73-7afd50d85e18"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:13:19.131699 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.131694 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/605bd53c-d414-4a19-ae73-7afd50d85e18-kube-api-access-vzj22" (OuterVolumeSpecName: "kube-api-access-vzj22") pod "605bd53c-d414-4a19-ae73-7afd50d85e18" (UID: "605bd53c-d414-4a19-ae73-7afd50d85e18"). InnerVolumeSpecName "kube-api-access-vzj22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:13:19.230134 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.230101 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/605bd53c-d414-4a19-ae73-7afd50d85e18-maas-api-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:19.230134 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.230129 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/605bd53c-d414-4a19-ae73-7afd50d85e18-openshift-service-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:19.230134 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.230138 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzj22\" (UniqueName: \"kubernetes.io/projected/605bd53c-d414-4a19-ae73-7afd50d85e18-kube-api-access-vzj22\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:19.503980 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.503948 2558 generic.go:356] "Generic (PLEG): container finished" podID="605bd53c-d414-4a19-ae73-7afd50d85e18" containerID="81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39" exitCode=0 Jul 07 00:13:19.504482 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.504011 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" Jul 07 00:13:19.504482 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.504038 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" event={"ID":"605bd53c-d414-4a19-ae73-7afd50d85e18","Type":"ContainerDied","Data":"81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39"} Jul 07 00:13:19.504482 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.504097 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs" event={"ID":"605bd53c-d414-4a19-ae73-7afd50d85e18","Type":"ContainerDied","Data":"9977646b2a1c34e370b8508c7e5d23c6968b2b383fd8917b166c2586ddc0ba21"} Jul 07 00:13:19.504482 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.504120 2558 scope.go:117] "RemoveContainer" containerID="81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39" Jul 07 00:13:19.515070 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.515041 2558 scope.go:117] "RemoveContainer" containerID="81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39" Jul 07 00:13:19.515553 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:13:19.515526 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39\": container with ID starting with 81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39 not found: ID does not exist" containerID="81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39" Jul 07 00:13:19.515620 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.515566 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39"} err="failed to get container status \"81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39\": rpc error: code = NotFound desc = could not find container \"81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39\": container with ID starting with 81c01d0f2eec214bb52dce0d5e2d5a503f9520aff4b8839c1b80872650e76e39 not found: ID does not exist" Jul 07 00:13:19.525734 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.525706 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs"] Jul 07 00:13:19.532467 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:19.532445 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-8cd6b865-6b9644c74f-zctvs"] Jul 07 00:13:20.050929 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:20.050898 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="605bd53c-d414-4a19-ae73-7afd50d85e18" path="/var/lib/kubelet/pods/605bd53c-d414-4a19-ae73-7afd50d85e18/volumes" Jul 07 00:13:24.529205 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:24.529168 2558 generic.go:356] "Generic (PLEG): container finished" podID="90564228-4e90-4549-ab63-e7e45c2687d2" containerID="6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8" exitCode=0 Jul 07 00:13:24.529623 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:24.529235 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" event={"ID":"90564228-4e90-4549-ab63-e7e45c2687d2","Type":"ContainerDied","Data":"6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8"} Jul 07 00:13:24.529667 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:24.529653 2558 scope.go:117] "RemoveContainer" containerID="6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8" Jul 07 00:13:24.530657 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:24.530629 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:13:24.530705 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:24.530697 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:13:24.530740 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:24.530731 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:13:24.570612 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:24.570577 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf"] Jul 07 00:13:24.589426 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:24.589394 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:24.690297 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:24.690255 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:25.535909 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:25.535876 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" event={"ID":"90564228-4e90-4549-ab63-e7e45c2687d2","Type":"ContainerStarted","Data":"96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572"} Jul 07 00:13:25.536360 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:25.536008 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" podUID="90564228-4e90-4549-ab63-e7e45c2687d2" containerName="istio-proxy" containerID="cri-o://96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572" gracePeriod=30 Jul 07 00:13:26.505800 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:26.505763 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:29.473382 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:29.473338 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:30.413206 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.413168 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:30.496509 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.496466 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:30.640102 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.640065 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:30.705690 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.705651 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:30.807516 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.807491 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:30.835703 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.835674 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl7vk\" (UniqueName: \"kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-kube-api-access-dl7vk\") pod \"90564228-4e90-4549-ab63-e7e45c2687d2\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " Jul 07 00:13:30.835892 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.835737 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90564228-4e90-4549-ab63-e7e45c2687d2-istiod-ca-cert\") pod \"90564228-4e90-4549-ab63-e7e45c2687d2\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " Jul 07 00:13:30.835892 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.835762 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-data\") pod \"90564228-4e90-4549-ab63-e7e45c2687d2\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " Jul 07 00:13:30.835892 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.835793 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-istio-token\") pod \"90564228-4e90-4549-ab63-e7e45c2687d2\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " Jul 07 00:13:30.835892 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.835814 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-credential-socket\") pod \"90564228-4e90-4549-ab63-e7e45c2687d2\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " Jul 07 00:13:30.835892 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.835879 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-certs\") pod \"90564228-4e90-4549-ab63-e7e45c2687d2\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " Jul 07 00:13:30.835892 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.835895 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-envoy\") pod \"90564228-4e90-4549-ab63-e7e45c2687d2\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " Jul 07 00:13:30.836215 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.835947 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90564228-4e90-4549-ab63-e7e45c2687d2-istio-podinfo\") pod \"90564228-4e90-4549-ab63-e7e45c2687d2\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " Jul 07 00:13:30.836215 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.835971 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-socket\") pod \"90564228-4e90-4549-ab63-e7e45c2687d2\" (UID: \"90564228-4e90-4549-ab63-e7e45c2687d2\") " Jul 07 00:13:30.836409 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.836381 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "90564228-4e90-4549-ab63-e7e45c2687d2" (UID: "90564228-4e90-4549-ab63-e7e45c2687d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:13:30.836607 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.836439 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "90564228-4e90-4549-ab63-e7e45c2687d2" (UID: "90564228-4e90-4549-ab63-e7e45c2687d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:13:30.836727 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.836475 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90564228-4e90-4549-ab63-e7e45c2687d2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "90564228-4e90-4549-ab63-e7e45c2687d2" (UID: "90564228-4e90-4549-ab63-e7e45c2687d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:13:30.836727 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.836651 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "90564228-4e90-4549-ab63-e7e45c2687d2" (UID: "90564228-4e90-4549-ab63-e7e45c2687d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:13:30.837033 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.837005 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-data" (OuterVolumeSpecName: "istio-data") pod "90564228-4e90-4549-ab63-e7e45c2687d2" (UID: "90564228-4e90-4549-ab63-e7e45c2687d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:13:30.838109 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.838074 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-kube-api-access-dl7vk" (OuterVolumeSpecName: "kube-api-access-dl7vk") pod "90564228-4e90-4549-ab63-e7e45c2687d2" (UID: "90564228-4e90-4549-ab63-e7e45c2687d2"). InnerVolumeSpecName "kube-api-access-dl7vk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:13:30.838666 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.838645 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "90564228-4e90-4549-ab63-e7e45c2687d2" (UID: "90564228-4e90-4549-ab63-e7e45c2687d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:13:30.838749 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.838695 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-istio-token" (OuterVolumeSpecName: "istio-token") pod "90564228-4e90-4549-ab63-e7e45c2687d2" (UID: "90564228-4e90-4549-ab63-e7e45c2687d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:13:30.838811 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.838791 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90564228-4e90-4549-ab63-e7e45c2687d2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "90564228-4e90-4549-ab63-e7e45c2687d2" (UID: "90564228-4e90-4549-ab63-e7e45c2687d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:13:30.936906 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.936835 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90564228-4e90-4549-ab63-e7e45c2687d2-istio-podinfo\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:30.936906 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.936895 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:30.936906 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.936905 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl7vk\" (UniqueName: \"kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-kube-api-access-dl7vk\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:30.936906 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.936916 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90564228-4e90-4549-ab63-e7e45c2687d2-istiod-ca-cert\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:30.937181 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.936925 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-data\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:30.937181 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.936932 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90564228-4e90-4549-ab63-e7e45c2687d2-istio-token\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:30.937181 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.936940 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-credential-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:30.937181 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.936947 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-workload-certs\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:30.937181 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:30.936956 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90564228-4e90-4549-ab63-e7e45c2687d2-istio-envoy\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:31.565868 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.565823 2558 generic.go:356] "Generic (PLEG): container finished" podID="90564228-4e90-4549-ab63-e7e45c2687d2" containerID="96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572" exitCode=0 Jul 07 00:13:31.566395 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.565903 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" Jul 07 00:13:31.566395 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.565908 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" event={"ID":"90564228-4e90-4549-ab63-e7e45c2687d2","Type":"ContainerDied","Data":"96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572"} Jul 07 00:13:31.566395 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.565980 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf" event={"ID":"90564228-4e90-4549-ab63-e7e45c2687d2","Type":"ContainerDied","Data":"8bfbb6829487f110364fd392d004603b6a7ec5658d0ffa8d9ec5f9d61404a4f8"} Jul 07 00:13:31.566395 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.566007 2558 scope.go:117] "RemoveContainer" containerID="96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572" Jul 07 00:13:31.576248 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.576221 2558 scope.go:117] "RemoveContainer" containerID="6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8" Jul 07 00:13:31.584694 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.584673 2558 scope.go:117] "RemoveContainer" containerID="96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572" Jul 07 00:13:31.585073 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:13:31.585053 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572\": container with ID starting with 96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572 not found: ID does not exist" containerID="96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572" Jul 07 00:13:31.585156 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.585085 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572"} err="failed to get container status \"96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572\": rpc error: code = NotFound desc = could not find container \"96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572\": container with ID starting with 96288213899a64d78b814efb62c83574ccb7732f6802f4d596305dff61126572 not found: ID does not exist" Jul 07 00:13:31.585156 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.585109 2558 scope.go:117] "RemoveContainer" containerID="6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8" Jul 07 00:13:31.585398 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:13:31.585380 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8\": container with ID starting with 6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8 not found: ID does not exist" containerID="6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8" Jul 07 00:13:31.585442 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.585405 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8"} err="failed to get container status \"6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8\": rpc error: code = NotFound desc = could not find container \"6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8\": container with ID starting with 6f3daf73245a3b1ec7193c14260f6407a06369e11063cc8ea0e2f3d215c4faa8 not found: ID does not exist" Jul 07 00:13:31.588331 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.588301 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf"] Jul 07 00:13:31.592452 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.592426 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8cd6b865-openshift-default-67d879d487-92rvf"] Jul 07 00:13:31.750776 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.750741 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:31.847930 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.847829 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:31.883440 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.883392 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh"] Jul 07 00:13:31.883948 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.883928 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90564228-4e90-4549-ab63-e7e45c2687d2" containerName="istio-proxy" Jul 07 00:13:31.884027 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.883952 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="90564228-4e90-4549-ab63-e7e45c2687d2" containerName="istio-proxy" Jul 07 00:13:31.884027 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.883965 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605bd53c-d414-4a19-ae73-7afd50d85e18" containerName="maas-api" Jul 07 00:13:31.884027 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.883973 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="605bd53c-d414-4a19-ae73-7afd50d85e18" containerName="maas-api" Jul 07 00:13:31.884027 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.883985 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90564228-4e90-4549-ab63-e7e45c2687d2" containerName="istio-proxy" Jul 07 00:13:31.884027 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.883994 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="90564228-4e90-4549-ab63-e7e45c2687d2" containerName="istio-proxy" Jul 07 00:13:31.884202 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.884073 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="90564228-4e90-4549-ab63-e7e45c2687d2" containerName="istio-proxy" Jul 07 00:13:31.884202 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.884082 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="90564228-4e90-4549-ab63-e7e45c2687d2" containerName="istio-proxy" Jul 07 00:13:31.884202 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.884093 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="605bd53c-d414-4a19-ae73-7afd50d85e18" containerName="maas-api" Jul 07 00:13:31.887322 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.887300 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:31.890509 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.890487 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-57dc1bd9-serving-cert\"" Jul 07 00:13:31.899988 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.899957 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh"] Jul 07 00:13:31.946144 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.946109 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj8rt\" (UniqueName: \"kubernetes.io/projected/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-kube-api-access-pj8rt\") pod \"maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:31.946334 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.946173 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-maas-api-tls\") pod \"maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:31.946334 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.946300 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-openshift-service-ca\") pod \"maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:31.950020 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:31.949991 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:32.046790 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.046762 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-maas-api-tls\") pod \"maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:32.046977 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.046820 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-openshift-service-ca\") pod \"maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:32.046977 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.046863 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj8rt\" (UniqueName: \"kubernetes.io/projected/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-kube-api-access-pj8rt\") pod \"maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:32.046977 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:13:32.046943 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-57dc1bd9-serving-cert: secret "maas-api-e2e-derive-57dc1bd9-serving-cert" not found Jul 07 00:13:32.047082 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:13:32.047034 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-maas-api-tls podName:79c9cdc4-c706-4a57-8fd0-8ead1c4d838e nodeName:}" failed. No retries permitted until 2026-07-07 00:13:32.547016994 +0000 UTC m=+1493.057177929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-maas-api-tls") pod "maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" (UID: "79c9cdc4-c706-4a57-8fd0-8ead1c4d838e") : secret "maas-api-e2e-derive-57dc1bd9-serving-cert" not found Jul 07 00:13:32.047708 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.047684 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-openshift-service-ca\") pod \"maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:32.052544 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.052516 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90564228-4e90-4549-ab63-e7e45c2687d2" path="/var/lib/kubelet/pods/90564228-4e90-4549-ab63-e7e45c2687d2/volumes" Jul 07 00:13:32.059070 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.059042 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj8rt\" (UniqueName: \"kubernetes.io/projected/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-kube-api-access-pj8rt\") pod \"maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:32.257801 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.257760 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:32.413459 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.413427 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:32.515333 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.515244 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:32.552005 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.551944 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-maas-api-tls\") pod \"maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:32.554779 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.554753 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-maas-api-tls\") pod \"maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:32.800152 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.800058 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:32.938954 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:32.938911 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh"] Jul 07 00:13:32.941351 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:13:32.941318 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c9cdc4_c706_4a57_8fd0_8ead1c4d838e.slice/crio-f6a05d8346667a9fd293ac4a0d9dc36699809e070119ad178cd303f069e2d1eb WatchSource:0}: Error finding container f6a05d8346667a9fd293ac4a0d9dc36699809e070119ad178cd303f069e2d1eb: Status 404 returned error can't find the container with id f6a05d8346667a9fd293ac4a0d9dc36699809e070119ad178cd303f069e2d1eb Jul 07 00:13:33.578897 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:33.578716 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" event={"ID":"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e","Type":"ContainerStarted","Data":"286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd"} Jul 07 00:13:33.579141 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:33.578908 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" event={"ID":"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e","Type":"ContainerStarted","Data":"f6a05d8346667a9fd293ac4a0d9dc36699809e070119ad178cd303f069e2d1eb"} Jul 07 00:13:33.579141 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:33.578944 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:33.596121 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:33.596069 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" podStartSLOduration=2.236734035 podStartE2EDuration="2.596053592s" totalImagesPullingTime="359.319557ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:13:33.595748068 +0000 UTC m=+1494.105909044" watchObservedRunningTime="2026-07-07 00:13:33.596053592 +0000 UTC m=+1494.106214549" Jul 07 00:13:33.990475 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:33.990442 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:36.243600 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.243566 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:36.476170 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.476129 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:36.584393 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.584287 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:36.666714 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.666681 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh"] Jul 07 00:13:36.666949 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.666913 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" podUID="79c9cdc4-c706-4a57-8fd0-8ead1c4d838e" containerName="maas-api" containerID="cri-o://286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd" gracePeriod=30 Jul 07 00:13:36.696236 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.696202 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:36.908585 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.908563 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:36.994366 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.994327 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj8rt\" (UniqueName: \"kubernetes.io/projected/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-kube-api-access-pj8rt\") pod \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " Jul 07 00:13:36.994366 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.994370 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-openshift-service-ca\") pod \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " Jul 07 00:13:36.994643 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.994452 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-maas-api-tls\") pod \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\" (UID: \"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e\") " Jul 07 00:13:36.994778 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.994756 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79c9cdc4-c706-4a57-8fd0-8ead1c4d838e" (UID: "79c9cdc4-c706-4a57-8fd0-8ead1c4d838e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:13:36.996833 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.996798 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79c9cdc4-c706-4a57-8fd0-8ead1c4d838e" (UID: "79c9cdc4-c706-4a57-8fd0-8ead1c4d838e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:13:36.996833 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:36.996823 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-kube-api-access-pj8rt" (OuterVolumeSpecName: "kube-api-access-pj8rt") pod "79c9cdc4-c706-4a57-8fd0-8ead1c4d838e" (UID: "79c9cdc4-c706-4a57-8fd0-8ead1c4d838e"). InnerVolumeSpecName "kube-api-access-pj8rt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:13:37.095487 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.095446 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-maas-api-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:37.095487 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.095480 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj8rt\" (UniqueName: \"kubernetes.io/projected/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-kube-api-access-pj8rt\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:37.095487 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.095494 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e-openshift-service-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:13:37.605262 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.605226 2558 generic.go:356] "Generic (PLEG): container finished" podID="79c9cdc4-c706-4a57-8fd0-8ead1c4d838e" containerID="286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd" exitCode=0 Jul 07 00:13:37.605708 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.605285 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" Jul 07 00:13:37.605708 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.605301 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" event={"ID":"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e","Type":"ContainerDied","Data":"286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd"} Jul 07 00:13:37.605708 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.605386 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh" event={"ID":"79c9cdc4-c706-4a57-8fd0-8ead1c4d838e","Type":"ContainerDied","Data":"f6a05d8346667a9fd293ac4a0d9dc36699809e070119ad178cd303f069e2d1eb"} Jul 07 00:13:37.605708 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.605412 2558 scope.go:117] "RemoveContainer" containerID="286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd" Jul 07 00:13:37.614651 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.614632 2558 scope.go:117] "RemoveContainer" containerID="286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd" Jul 07 00:13:37.614952 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:13:37.614928 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd\": container with ID starting with 286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd not found: ID does not exist" containerID="286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd" Jul 07 00:13:37.615054 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.614958 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd"} err="failed to get container status \"286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd\": rpc error: code = NotFound desc = could not find container \"286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd\": container with ID starting with 286ad16ed4f15abe00570fff9dc8b6eb09ecc73114bad81d2369fa6e49aa28dd not found: ID does not exist" Jul 07 00:13:37.627669 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.627636 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh"] Jul 07 00:13:37.631118 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.631093 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-57dc1bd9-677cb8d74c-sgszh"] Jul 07 00:13:37.895971 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:37.895922 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:38.052943 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:38.052905 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79c9cdc4-c706-4a57-8fd0-8ead1c4d838e" path="/var/lib/kubelet/pods/79c9cdc4-c706-4a57-8fd0-8ead1c4d838e/volumes" Jul 07 00:13:40.048715 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:40.048687 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:13:40.052153 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:40.052130 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:13:42.516357 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:42.516317 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:42.618210 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:42.618173 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:43.047803 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:43.047769 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:43.141139 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:43.141092 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:43.241256 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:43.241192 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:44.728137 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:44.728096 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:51.133898 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:51.133865 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:51.235233 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:13:51.235192 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:17.140484 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:17.140450 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:21.405813 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:21.405751 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:22.977396 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:22.977359 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:23.055957 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:23.055890 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:23.169380 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:23.169331 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:39.121532 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:39.121498 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:45.140086 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:45.140047 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:45.208430 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:45.208386 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:45.282573 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:45.282525 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:45.383479 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:45.383441 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:55.916828 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:55.916792 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:56.772170 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:14:56.772131 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:02.477595 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:02.477562 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:02.545551 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:02.545510 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:02.664690 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:02.664652 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:02.851168 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:02.851085 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:02.949136 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:02.949094 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:03.639746 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:03.639711 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:03.722181 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:03.722141 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:03.824029 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:03.823993 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:05.565789 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:05.565752 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:11.475675 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:11.475634 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:11.538001 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:11.537963 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:12.192308 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:12.192273 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:18.110877 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.110815 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:18.211318 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.211281 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:18.360469 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.360414 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:18.777963 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.777924 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb"] Jul 07 00:15:18.778330 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.778312 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c9cdc4-c706-4a57-8fd0-8ead1c4d838e" containerName="maas-api" Jul 07 00:15:18.778420 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.778333 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c9cdc4-c706-4a57-8fd0-8ead1c4d838e" containerName="maas-api" Jul 07 00:15:18.778475 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.778418 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c9cdc4-c706-4a57-8fd0-8ead1c4d838e" containerName="maas-api" Jul 07 00:15:18.781284 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.781260 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.784157 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.784134 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-16542ae4-openshift-default-dockercfg-fpxsd\"" Jul 07 00:15:18.796786 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.796756 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb"] Jul 07 00:15:18.810904 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.810845 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:18.887737 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.887697 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/236d8063-04de-44e5-a670-b86bd1db3932-istiod-ca-cert\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.887737 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.887737 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-data\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.887991 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.887757 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7vd2\" (UniqueName: \"kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-kube-api-access-d7vd2\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.887991 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.887817 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-certs\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.887991 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.887887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-socket\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.887991 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.887908 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-credential-socket\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.887991 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.887933 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-envoy\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.887991 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.887959 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-istio-token\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.888184 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.888008 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/236d8063-04de-44e5-a670-b86bd1db3932-istio-podinfo\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.988651 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.988613 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-socket\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.988651 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.988651 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-credential-socket\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.988921 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.988668 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-envoy\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.988921 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.988750 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-istio-token\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.988921 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.988828 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/236d8063-04de-44e5-a670-b86bd1db3932-istio-podinfo\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.988921 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.988897 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/236d8063-04de-44e5-a670-b86bd1db3932-istiod-ca-cert\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.989112 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.988933 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-data\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.989112 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.989085 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7vd2\" (UniqueName: \"kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-kube-api-access-d7vd2\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.989222 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.989137 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-certs\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.989222 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.989202 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-credential-socket\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.989340 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.989251 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-socket\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.989502 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.989477 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-data\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.989800 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.989528 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-certs\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.989800 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.989664 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/236d8063-04de-44e5-a670-b86bd1db3932-istiod-ca-cert\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.991137 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.991115 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-envoy\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:18.991495 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.991478 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/236d8063-04de-44e5-a670-b86bd1db3932-istio-podinfo\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:19.000015 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:18.999985 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7vd2\" (UniqueName: \"kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-kube-api-access-d7vd2\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:19.000132 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:19.000071 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-istio-token\") pod \"e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:19.068276 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:19.068174 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:19.091952 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:19.091921 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:19.238728 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:19.238702 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb"] Jul 07 00:15:19.241352 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:15:19.241309 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236d8063_04de_44e5_a670_b86bd1db3932.slice/crio-57a6e415a40cb07ecdae58e697f76943938666b8fc016316e321cb03adbe6dbd WatchSource:0}: Error finding container 57a6e415a40cb07ecdae58e697f76943938666b8fc016316e321cb03adbe6dbd: Status 404 returned error can't find the container with id 57a6e415a40cb07ecdae58e697f76943938666b8fc016316e321cb03adbe6dbd Jul 07 00:15:19.243559 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:19.243526 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:15:19.243625 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:19.243593 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:15:19.243677 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:19.243624 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:15:20.063772 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:20.063731 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" event={"ID":"236d8063-04de-44e5-a670-b86bd1db3932","Type":"ContainerStarted","Data":"31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9"} Jul 07 00:15:20.063989 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:20.063828 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" event={"ID":"236d8063-04de-44e5-a670-b86bd1db3932","Type":"ContainerStarted","Data":"57a6e415a40cb07ecdae58e697f76943938666b8fc016316e321cb03adbe6dbd"} Jul 07 00:15:20.086287 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:20.086228 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" podStartSLOduration=2.086211544 podStartE2EDuration="2.086211544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:20.08459865 +0000 UTC m=+1600.594759607" watchObservedRunningTime="2026-07-07 00:15:20.086211544 +0000 UTC m=+1600.596372500" Jul 07 00:15:20.092693 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:20.092664 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:20.097797 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:20.097770 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:21.068781 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:21.068747 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:21.069816 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:21.069801 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:22.047677 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.047637 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:22.098478 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.098443 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl"] Jul 07 00:15:22.101978 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.101951 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:22.104565 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.104538 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-16542ae4-serving-cert\"" Jul 07 00:15:22.114413 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.114383 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl"] Jul 07 00:15:22.132204 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.132161 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:22.203327 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.203297 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:22.220221 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.220182 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-maas-api-tls\") pod \"maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:22.220221 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.220222 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-openshift-service-ca\") pod \"maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:22.220559 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.220357 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5tzt\" (UniqueName: \"kubernetes.io/projected/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-kube-api-access-c5tzt\") pod \"maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:22.321598 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.321500 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5tzt\" (UniqueName: \"kubernetes.io/projected/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-kube-api-access-c5tzt\") pod \"maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:22.321598 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.321573 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-maas-api-tls\") pod \"maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:22.321598 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.321591 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-openshift-service-ca\") pod \"maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:22.321830 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:15:22.321715 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-16542ae4-serving-cert: secret "maas-api-e2e-mt-16542ae4-serving-cert" not found Jul 07 00:15:22.321830 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:15:22.321786 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-maas-api-tls podName:7e6e1ea0-7cf7-4364-b59e-0eeab4007229 nodeName:}" failed. No retries permitted until 2026-07-07 00:15:22.821768114 +0000 UTC m=+1603.331929055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-maas-api-tls") pod "maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" (UID: "7e6e1ea0-7cf7-4364-b59e-0eeab4007229") : secret "maas-api-e2e-mt-16542ae4-serving-cert" not found Jul 07 00:15:22.322297 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.322278 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-openshift-service-ca\") pod \"maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:22.333954 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.333917 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5tzt\" (UniqueName: \"kubernetes.io/projected/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-kube-api-access-c5tzt\") pod \"maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:22.401815 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.401776 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:22.601449 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.601364 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:22.704745 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.704710 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:22.827012 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.826975 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-maas-api-tls\") pod \"maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:22.829537 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:22.829507 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-maas-api-tls\") pod \"maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:23.015043 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:23.015001 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:23.159378 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:23.159304 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl"] Jul 07 00:15:23.161549 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:15:23.161518 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6e1ea0_7cf7_4364_b59e_0eeab4007229.slice/crio-24e372e6fa0482274f617002d0974ffbd040356543e7e6e142236cbf27bc6f93 WatchSource:0}: Error finding container 24e372e6fa0482274f617002d0974ffbd040356543e7e6e142236cbf27bc6f93: Status 404 returned error can't find the container with id 24e372e6fa0482274f617002d0974ffbd040356543e7e6e142236cbf27bc6f93 Jul 07 00:15:24.086037 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:24.085997 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" event={"ID":"7e6e1ea0-7cf7-4364-b59e-0eeab4007229","Type":"ContainerStarted","Data":"27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275"} Jul 07 00:15:24.086264 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:24.086062 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" event={"ID":"7e6e1ea0-7cf7-4364-b59e-0eeab4007229","Type":"ContainerStarted","Data":"24e372e6fa0482274f617002d0974ffbd040356543e7e6e142236cbf27bc6f93"} Jul 07 00:15:24.086264 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:24.086181 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:24.104694 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:24.104647 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" podStartSLOduration=1.6645567959999998 podStartE2EDuration="2.104629174s" totalImagesPullingTime="440.072378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:24.103475571 +0000 UTC m=+1604.613636529" watchObservedRunningTime="2026-07-07 00:15:24.104629174 +0000 UTC m=+1604.614790131" Jul 07 00:15:26.582191 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:26.582154 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:27.263218 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:27.263184 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:27.363156 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:27.363119 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:27.725460 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:27.725423 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:30.106296 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:30.106264 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:33.728985 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:33.728944 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:33.839444 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:33.839402 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:34.160648 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.160554 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:34.604595 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.604523 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl"] Jul 07 00:15:34.604869 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.604824 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" podUID="7e6e1ea0-7cf7-4364-b59e-0eeab4007229" containerName="maas-api" containerID="cri-o://27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275" gracePeriod=30 Jul 07 00:15:34.629648 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.629617 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:34.684609 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.684575 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:34.859455 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.859387 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:34.946559 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.946522 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-maas-api-tls\") pod \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " Jul 07 00:15:34.946559 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.946563 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5tzt\" (UniqueName: \"kubernetes.io/projected/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-kube-api-access-c5tzt\") pod \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " Jul 07 00:15:34.946764 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.946611 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-openshift-service-ca\") pod \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\" (UID: \"7e6e1ea0-7cf7-4364-b59e-0eeab4007229\") " Jul 07 00:15:34.947098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.947071 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7e6e1ea0-7cf7-4364-b59e-0eeab4007229" (UID: "7e6e1ea0-7cf7-4364-b59e-0eeab4007229"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:15:34.948791 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.948762 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7e6e1ea0-7cf7-4364-b59e-0eeab4007229" (UID: "7e6e1ea0-7cf7-4364-b59e-0eeab4007229"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:15:34.948912 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:34.948886 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-kube-api-access-c5tzt" (OuterVolumeSpecName: "kube-api-access-c5tzt") pod "7e6e1ea0-7cf7-4364-b59e-0eeab4007229" (UID: "7e6e1ea0-7cf7-4364-b59e-0eeab4007229"). InnerVolumeSpecName "kube-api-access-c5tzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:35.047655 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.047613 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-maas-api-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:35.047655 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.047643 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5tzt\" (UniqueName: \"kubernetes.io/projected/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-kube-api-access-c5tzt\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:35.047655 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.047661 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e6e1ea0-7cf7-4364-b59e-0eeab4007229-openshift-service-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:35.144867 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.144768 2558 generic.go:356] "Generic (PLEG): container finished" podID="7e6e1ea0-7cf7-4364-b59e-0eeab4007229" containerID="27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275" exitCode=0 Jul 07 00:15:35.144867 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.144836 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" Jul 07 00:15:35.145058 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.144877 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" event={"ID":"7e6e1ea0-7cf7-4364-b59e-0eeab4007229","Type":"ContainerDied","Data":"27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275"} Jul 07 00:15:35.145058 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.144933 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl" event={"ID":"7e6e1ea0-7cf7-4364-b59e-0eeab4007229","Type":"ContainerDied","Data":"24e372e6fa0482274f617002d0974ffbd040356543e7e6e142236cbf27bc6f93"} Jul 07 00:15:35.145058 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.144957 2558 scope.go:117] "RemoveContainer" containerID="27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275" Jul 07 00:15:35.154532 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.154450 2558 scope.go:117] "RemoveContainer" containerID="27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275" Jul 07 00:15:35.156393 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:15:35.156367 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275\": container with ID starting with 27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275 not found: ID does not exist" containerID="27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275" Jul 07 00:15:35.156506 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.156397 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275"} err="failed to get container status \"27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275\": rpc error: code = NotFound desc = could not find container \"27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275\": container with ID starting with 27338e0e86f55816382d8ffe0881019aa14ecd370a37396aba576e576afd9275 not found: ID does not exist" Jul 07 00:15:35.170611 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.170572 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl"] Jul 07 00:15:35.174304 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:35.174282 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-16542ae4-68c5fc6fdd-wbmtl"] Jul 07 00:15:36.051864 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:36.051816 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e6e1ea0-7cf7-4364-b59e-0eeab4007229" path="/var/lib/kubelet/pods/7e6e1ea0-7cf7-4364-b59e-0eeab4007229/volumes" Jul 07 00:15:36.187289 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:36.187256 2558 patch_prober.go:28] interesting pod/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:41934->10.132.0.40:15021: read: connection reset by peer" start-of-body= Jul 07 00:15:36.187404 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:36.187332 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" podUID="236d8063-04de-44e5-a670-b86bd1db3932" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:41934->10.132.0.40:15021: read: connection reset by peer" Jul 07 00:15:37.157208 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:37.157171 2558 generic.go:356] "Generic (PLEG): container finished" podID="236d8063-04de-44e5-a670-b86bd1db3932" containerID="31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9" exitCode=0 Jul 07 00:15:37.157649 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:37.157248 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" event={"ID":"236d8063-04de-44e5-a670-b86bd1db3932","Type":"ContainerDied","Data":"31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9"} Jul 07 00:15:37.157649 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:37.157644 2558 scope.go:117] "RemoveContainer" containerID="31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9" Jul 07 00:15:37.158593 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:37.158566 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:15:37.158655 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:37.158638 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:15:37.158691 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:37.158673 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:15:38.163769 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:38.163735 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" event={"ID":"236d8063-04de-44e5-a670-b86bd1db3932","Type":"ContainerStarted","Data":"9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970"} Jul 07 00:15:39.092791 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:39.092746 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:39.093011 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:39.092908 2558 patch_prober.go:28] interesting pod/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 00:15:39.093011 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:39.092952 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" podUID="236d8063-04de-44e5-a670-b86bd1db3932" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 00:15:40.092822 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:40.092776 2558 patch_prober.go:28] interesting pod/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 00:15:40.093356 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:40.092879 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" podUID="236d8063-04de-44e5-a670-b86bd1db3932" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 00:15:40.717674 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:40.717640 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:40.789952 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:40.789905 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:40.958338 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:40.958305 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb"] Jul 07 00:15:40.958551 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:40.958527 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" podUID="236d8063-04de-44e5-a670-b86bd1db3932" containerName="istio-proxy" containerID="cri-o://9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970" gracePeriod=30 Jul 07 00:15:40.986915 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:40.986792 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:41.204713 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:41.204680 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:41.346522 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:41.346421 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:41.439997 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:41.439956 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:41.574088 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:41.574044 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:47.271050 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.271011 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:47.351927 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.351866 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:47.407518 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.407493 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:47.465311 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465225 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7vd2\" (UniqueName: \"kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-kube-api-access-d7vd2\") pod \"236d8063-04de-44e5-a670-b86bd1db3932\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " Jul 07 00:15:47.465311 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465266 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-socket\") pod \"236d8063-04de-44e5-a670-b86bd1db3932\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " Jul 07 00:15:47.465556 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465321 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-istio-token\") pod \"236d8063-04de-44e5-a670-b86bd1db3932\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " Jul 07 00:15:47.465556 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465355 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/236d8063-04de-44e5-a670-b86bd1db3932-istiod-ca-cert\") pod \"236d8063-04de-44e5-a670-b86bd1db3932\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " Jul 07 00:15:47.465556 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465382 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-certs\") pod \"236d8063-04de-44e5-a670-b86bd1db3932\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " Jul 07 00:15:47.465720 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465577 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-credential-socket\") pod \"236d8063-04de-44e5-a670-b86bd1db3932\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " Jul 07 00:15:47.465720 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465628 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/236d8063-04de-44e5-a670-b86bd1db3932-istio-podinfo\") pod \"236d8063-04de-44e5-a670-b86bd1db3932\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " Jul 07 00:15:47.465720 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465671 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "236d8063-04de-44e5-a670-b86bd1db3932" (UID: "236d8063-04de-44e5-a670-b86bd1db3932"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:47.465720 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465680 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "236d8063-04de-44e5-a670-b86bd1db3932" (UID: "236d8063-04de-44e5-a670-b86bd1db3932"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:47.465720 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465702 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-data\") pod \"236d8063-04de-44e5-a670-b86bd1db3932\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " Jul 07 00:15:47.466020 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465732 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-envoy\") pod \"236d8063-04de-44e5-a670-b86bd1db3932\" (UID: \"236d8063-04de-44e5-a670-b86bd1db3932\") " Jul 07 00:15:47.466020 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465832 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "236d8063-04de-44e5-a670-b86bd1db3932" (UID: "236d8063-04de-44e5-a670-b86bd1db3932"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:47.466020 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.465873 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/236d8063-04de-44e5-a670-b86bd1db3932-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "236d8063-04de-44e5-a670-b86bd1db3932" (UID: "236d8063-04de-44e5-a670-b86bd1db3932"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:15:47.466170 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.466098 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/236d8063-04de-44e5-a670-b86bd1db3932-istiod-ca-cert\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:47.466170 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.466118 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-certs\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:47.466170 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.466131 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-credential-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:47.466170 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.466145 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-workload-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:47.466903 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.466878 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-data" (OuterVolumeSpecName: "istio-data") pod "236d8063-04de-44e5-a670-b86bd1db3932" (UID: "236d8063-04de-44e5-a670-b86bd1db3932"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:47.467677 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.467654 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-kube-api-access-d7vd2" (OuterVolumeSpecName: "kube-api-access-d7vd2") pod "236d8063-04de-44e5-a670-b86bd1db3932" (UID: "236d8063-04de-44e5-a670-b86bd1db3932"). InnerVolumeSpecName "kube-api-access-d7vd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:47.467955 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.467935 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/236d8063-04de-44e5-a670-b86bd1db3932-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "236d8063-04de-44e5-a670-b86bd1db3932" (UID: "236d8063-04de-44e5-a670-b86bd1db3932"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:15:47.468368 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.468343 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-istio-token" (OuterVolumeSpecName: "istio-token") pod "236d8063-04de-44e5-a670-b86bd1db3932" (UID: "236d8063-04de-44e5-a670-b86bd1db3932"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:47.468368 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.468355 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "236d8063-04de-44e5-a670-b86bd1db3932" (UID: "236d8063-04de-44e5-a670-b86bd1db3932"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:47.472983 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.472454 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:47.567047 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.567012 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-istio-token\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:47.567047 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.567038 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/236d8063-04de-44e5-a670-b86bd1db3932-istio-podinfo\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:47.567047 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.567048 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-data\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:47.567296 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.567058 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/236d8063-04de-44e5-a670-b86bd1db3932-istio-envoy\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:47.567296 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.567067 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7vd2\" (UniqueName: \"kubernetes.io/projected/236d8063-04de-44e5-a670-b86bd1db3932-kube-api-access-d7vd2\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:15:47.705297 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:47.705261 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:48.211052 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.211015 2558 generic.go:356] "Generic (PLEG): container finished" podID="236d8063-04de-44e5-a670-b86bd1db3932" containerID="9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970" exitCode=0 Jul 07 00:15:48.211266 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.211073 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" Jul 07 00:15:48.211266 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.211097 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" event={"ID":"236d8063-04de-44e5-a670-b86bd1db3932","Type":"ContainerDied","Data":"9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970"} Jul 07 00:15:48.211266 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.211153 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb" event={"ID":"236d8063-04de-44e5-a670-b86bd1db3932","Type":"ContainerDied","Data":"57a6e415a40cb07ecdae58e697f76943938666b8fc016316e321cb03adbe6dbd"} Jul 07 00:15:48.211266 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.211175 2558 scope.go:117] "RemoveContainer" containerID="9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970" Jul 07 00:15:48.219718 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.219700 2558 scope.go:117] "RemoveContainer" containerID="31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9" Jul 07 00:15:48.227599 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.227581 2558 scope.go:117] "RemoveContainer" containerID="9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970" Jul 07 00:15:48.227862 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:15:48.227827 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970\": container with ID starting with 9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970 not found: ID does not exist" containerID="9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970" Jul 07 00:15:48.227907 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.227876 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970"} err="failed to get container status \"9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970\": rpc error: code = NotFound desc = could not find container \"9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970\": container with ID starting with 9ba91f7aea5f59bc6f09921bc81e52929e194aaa1dcaa29a1105dacf8823a970 not found: ID does not exist" Jul 07 00:15:48.227964 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.227907 2558 scope.go:117] "RemoveContainer" containerID="31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9" Jul 07 00:15:48.228137 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:15:48.228119 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9\": container with ID starting with 31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9 not found: ID does not exist" containerID="31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9" Jul 07 00:15:48.228187 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.228148 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9"} err="failed to get container status \"31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9\": rpc error: code = NotFound desc = could not find container \"31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9\": container with ID starting with 31bb59b2b1ce64f29b183b5a4b45d67627b92e8bd629bbac20c5f01f9553bed9 not found: ID does not exist" Jul 07 00:15:48.231287 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.231266 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb"] Jul 07 00:15:48.234685 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:48.234661 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-16542ae4-openshift-default-6cf5c59467-gvcpb"] Jul 07 00:15:50.051527 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:50.051496 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="236d8063-04de-44e5-a670-b86bd1db3932" path="/var/lib/kubelet/pods/236d8063-04de-44e5-a670-b86bd1db3932/volumes" Jul 07 00:15:53.434203 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:53.434165 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:53.534310 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:15:53.534260 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:00.454213 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.454160 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:00.746577 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.746495 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p"] Jul 07 00:16:00.747029 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.747009 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="236d8063-04de-44e5-a670-b86bd1db3932" containerName="istio-proxy" Jul 07 00:16:00.747098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.747033 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="236d8063-04de-44e5-a670-b86bd1db3932" containerName="istio-proxy" Jul 07 00:16:00.747098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.747050 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e6e1ea0-7cf7-4364-b59e-0eeab4007229" containerName="maas-api" Jul 07 00:16:00.747098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.747058 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e6e1ea0-7cf7-4364-b59e-0eeab4007229" containerName="maas-api" Jul 07 00:16:00.747098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.747080 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="236d8063-04de-44e5-a670-b86bd1db3932" containerName="istio-proxy" Jul 07 00:16:00.747098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.747089 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="236d8063-04de-44e5-a670-b86bd1db3932" containerName="istio-proxy" Jul 07 00:16:00.747257 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.747204 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e6e1ea0-7cf7-4364-b59e-0eeab4007229" containerName="maas-api" Jul 07 00:16:00.747257 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.747217 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="236d8063-04de-44e5-a670-b86bd1db3932" containerName="istio-proxy" Jul 07 00:16:00.747257 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.747228 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="236d8063-04de-44e5-a670-b86bd1db3932" containerName="istio-proxy" Jul 07 00:16:00.750618 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.750583 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.753328 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.753308 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-104fd30e-openshift-default-dockercfg-mtbbd\"" Jul 07 00:16:00.765047 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.765020 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p"] Jul 07 00:16:00.788911 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.788844 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:00.886253 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.886215 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-podinfo\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.886253 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.886259 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-token\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.886483 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.886282 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-certs\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.886483 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.886355 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-credential-socket\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.886483 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.886402 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-envoy\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.886483 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.886443 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgvlf\" (UniqueName: \"kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-kube-api-access-mgvlf\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.886483 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.886482 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istiod-ca-cert\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.886642 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.886514 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-data\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.886642 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.886539 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-socket\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.986983 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.986940 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-credential-socket\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.986983 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.986985 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-envoy\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987256 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987031 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgvlf\" (UniqueName: \"kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-kube-api-access-mgvlf\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987256 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987063 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istiod-ca-cert\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987256 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987103 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-data\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987256 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987125 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-socket\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987256 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987166 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-podinfo\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987256 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987186 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-token\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987256 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987210 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-certs\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987631 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987448 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-credential-socket\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987631 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987559 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-data\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987812 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987787 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istiod-ca-cert\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987912 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987831 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-socket\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.987912 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.987887 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-certs\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.989428 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.989406 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-envoy\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.989720 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.989703 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-podinfo\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.991109 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.991082 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:00.994919 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.994893 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-token\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:00.995045 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:00.995029 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgvlf\" (UniqueName: \"kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-kube-api-access-mgvlf\") pod \"e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:01.061529 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:01.061423 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:01.201871 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:01.201817 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p"] Jul 07 00:16:01.204525 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:16:01.204496 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d3d593_5fbe_468a_98e2_e1e39fa382f7.slice/crio-31f586776a28d0f3d113ecbb672a88768ca3e281843a0e165f0aeea3ca99ae8f WatchSource:0}: Error finding container 31f586776a28d0f3d113ecbb672a88768ca3e281843a0e165f0aeea3ca99ae8f: Status 404 returned error can't find the container with id 31f586776a28d0f3d113ecbb672a88768ca3e281843a0e165f0aeea3ca99ae8f Jul 07 00:16:01.206798 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:01.206764 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:16:01.206938 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:01.206869 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:16:01.206938 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:01.206902 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:16:01.270077 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:01.270042 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" event={"ID":"76d3d593-5fbe-468a-98e2-e1e39fa382f7","Type":"ContainerStarted","Data":"58d4c1f11ff5391918f081edb95d8c6c8535dfdc81997b5da47a3e0395445440"} Jul 07 00:16:01.270245 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:01.270100 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" event={"ID":"76d3d593-5fbe-468a-98e2-e1e39fa382f7","Type":"ContainerStarted","Data":"31f586776a28d0f3d113ecbb672a88768ca3e281843a0e165f0aeea3ca99ae8f"} Jul 07 00:16:01.290649 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:01.290595 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" podStartSLOduration=1.290578854 podStartE2EDuration="1.290578854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:16:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:16:01.28888279 +0000 UTC m=+1641.799043745" watchObservedRunningTime="2026-07-07 00:16:01.290578854 +0000 UTC m=+1641.800739810" Jul 07 00:16:02.062512 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:02.062475 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:02.067720 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:02.067694 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:02.274115 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:02.274080 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:02.275131 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:02.275106 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:03.250721 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.250680 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:03.302737 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.302701 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k"] Jul 07 00:16:03.305261 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.305243 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.307948 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.307921 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-104fd30e-serving-cert\"" Jul 07 00:16:03.314514 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.314471 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k"] Jul 07 00:16:03.335576 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.335536 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:03.406586 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.406545 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:03.408713 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.408677 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb1bb15-2b83-4698-801c-6f974c940753-openshift-service-ca\") pod \"maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.408965 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.408937 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5d67\" (UniqueName: \"kubernetes.io/projected/0cb1bb15-2b83-4698-801c-6f974c940753-kube-api-access-v5d67\") pod \"maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.409048 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.409020 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb1bb15-2b83-4698-801c-6f974c940753-maas-api-tls\") pod \"maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.510058 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.509968 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5d67\" (UniqueName: \"kubernetes.io/projected/0cb1bb15-2b83-4698-801c-6f974c940753-kube-api-access-v5d67\") pod \"maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.510058 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.510017 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb1bb15-2b83-4698-801c-6f974c940753-maas-api-tls\") pod \"maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.510058 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.510056 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb1bb15-2b83-4698-801c-6f974c940753-openshift-service-ca\") pod \"maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.510901 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.510814 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb1bb15-2b83-4698-801c-6f974c940753-openshift-service-ca\") pod \"maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.511460 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.511440 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:03.512867 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.512834 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb1bb15-2b83-4698-801c-6f974c940753-maas-api-tls\") pod \"maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.519091 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.519066 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5d67\" (UniqueName: \"kubernetes.io/projected/0cb1bb15-2b83-4698-801c-6f974c940753-kube-api-access-v5d67\") pod \"maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.617386 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.617344 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:03.753902 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.753871 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k"] Jul 07 00:16:03.756731 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:16:03.756701 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb1bb15_2b83_4698_801c_6f974c940753.slice/crio-0ce97ee01a553a8ab302fa07f89c468940cc7e9651b64d5abf94ca28d00639d0 WatchSource:0}: Error finding container 0ce97ee01a553a8ab302fa07f89c468940cc7e9651b64d5abf94ca28d00639d0: Status 404 returned error can't find the container with id 0ce97ee01a553a8ab302fa07f89c468940cc7e9651b64d5abf94ca28d00639d0 Jul 07 00:16:03.890155 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.890117 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:03.992035 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:03.991998 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:04.284330 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:04.284294 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" event={"ID":"0cb1bb15-2b83-4698-801c-6f974c940753","Type":"ContainerStarted","Data":"a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5"} Jul 07 00:16:04.284783 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:04.284349 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" event={"ID":"0cb1bb15-2b83-4698-801c-6f974c940753","Type":"ContainerStarted","Data":"0ce97ee01a553a8ab302fa07f89c468940cc7e9651b64d5abf94ca28d00639d0"} Jul 07 00:16:04.302136 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:04.302075 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" podStartSLOduration=0.949789416 podStartE2EDuration="1.302056795s" totalImagesPullingTime="352.267379ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:16:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:16:04.302032717 +0000 UTC m=+1644.812193674" watchObservedRunningTime="2026-07-07 00:16:04.302056795 +0000 UTC m=+1644.812217751" Jul 07 00:16:05.289225 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:05.289194 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:07.784032 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:07.783995 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:11.302596 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:11.302556 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:16.403676 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:16.403626 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:17.978292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:17.977791 2558 patch_prober.go:28] interesting pod/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:56032->10.132.0.42:15021: read: connection reset by peer" start-of-body= Jul 07 00:16:17.978292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:17.977875 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:56032->10.132.0.42:15021: read: connection reset by peer" Jul 07 00:16:18.350032 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:18.349943 2558 generic.go:356] "Generic (PLEG): container finished" podID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerID="58d4c1f11ff5391918f081edb95d8c6c8535dfdc81997b5da47a3e0395445440" exitCode=0 Jul 07 00:16:18.350032 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:18.350015 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" event={"ID":"76d3d593-5fbe-468a-98e2-e1e39fa382f7","Type":"ContainerDied","Data":"58d4c1f11ff5391918f081edb95d8c6c8535dfdc81997b5da47a3e0395445440"} Jul 07 00:16:18.350429 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:18.350414 2558 scope.go:117] "RemoveContainer" containerID="58d4c1f11ff5391918f081edb95d8c6c8535dfdc81997b5da47a3e0395445440" Jul 07 00:16:18.351432 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:18.351398 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:16:18.351493 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:18.351475 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:16:18.351530 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:18.351514 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:16:19.356419 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:19.356381 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" event={"ID":"76d3d593-5fbe-468a-98e2-e1e39fa382f7","Type":"ContainerStarted","Data":"20ee59b74b923d38890af39995fd24dfdf33f77a5418c18c4f6f0a6b01aeb1b9"} Jul 07 00:16:20.062090 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:20.062056 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:20.062311 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:20.062289 2558 patch_prober.go:28] interesting pod/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 00:16:20.062358 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:20.062340 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 00:16:21.062256 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:21.062215 2558 patch_prober.go:28] interesting pod/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 00:16:21.062643 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:21.062277 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 00:16:22.062729 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:22.062692 2558 patch_prober.go:28] interesting pod/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 00:16:22.063235 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:22.062774 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 00:16:23.062154 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:23.062114 2558 patch_prober.go:28] interesting pod/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 00:16:23.062348 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:23.062176 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 00:16:24.065153 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:24.065125 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:24.065644 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:24.065489 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:24.066200 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:24.066178 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:16:26.124004 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:26.123973 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:34.550307 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:34.550249 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:34.649475 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:34.649440 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:40.271558 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:40.271512 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:40.715672 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:40.715626 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:40.816765 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:40.816728 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:40.929367 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:40.929327 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k"] Jul 07 00:16:40.929647 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:40.929615 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" podUID="0cb1bb15-2b83-4698-801c-6f974c940753" containerName="maas-api" containerID="cri-o://a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5" gracePeriod=30 Jul 07 00:16:40.962725 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:40.962698 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:41.178298 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.178264 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:41.265603 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.265567 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb1bb15-2b83-4698-801c-6f974c940753-maas-api-tls\") pod \"0cb1bb15-2b83-4698-801c-6f974c940753\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " Jul 07 00:16:41.265795 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.265665 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb1bb15-2b83-4698-801c-6f974c940753-openshift-service-ca\") pod \"0cb1bb15-2b83-4698-801c-6f974c940753\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " Jul 07 00:16:41.265795 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.265713 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5d67\" (UniqueName: \"kubernetes.io/projected/0cb1bb15-2b83-4698-801c-6f974c940753-kube-api-access-v5d67\") pod \"0cb1bb15-2b83-4698-801c-6f974c940753\" (UID: \"0cb1bb15-2b83-4698-801c-6f974c940753\") " Jul 07 00:16:41.266080 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.266055 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb1bb15-2b83-4698-801c-6f974c940753-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0cb1bb15-2b83-4698-801c-6f974c940753" (UID: "0cb1bb15-2b83-4698-801c-6f974c940753"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:16:41.267922 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.267888 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cb1bb15-2b83-4698-801c-6f974c940753-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0cb1bb15-2b83-4698-801c-6f974c940753" (UID: "0cb1bb15-2b83-4698-801c-6f974c940753"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:16:41.268019 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.267985 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb1bb15-2b83-4698-801c-6f974c940753-kube-api-access-v5d67" (OuterVolumeSpecName: "kube-api-access-v5d67") pod "0cb1bb15-2b83-4698-801c-6f974c940753" (UID: "0cb1bb15-2b83-4698-801c-6f974c940753"). InnerVolumeSpecName "kube-api-access-v5d67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:16:41.366689 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.366585 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb1bb15-2b83-4698-801c-6f974c940753-maas-api-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:16:41.366689 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.366632 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb1bb15-2b83-4698-801c-6f974c940753-openshift-service-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:16:41.366689 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.366643 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5d67\" (UniqueName: \"kubernetes.io/projected/0cb1bb15-2b83-4698-801c-6f974c940753-kube-api-access-v5d67\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:16:41.475062 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.475024 2558 generic.go:356] "Generic (PLEG): container finished" podID="0cb1bb15-2b83-4698-801c-6f974c940753" containerID="a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5" exitCode=0 Jul 07 00:16:41.475062 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.475064 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" event={"ID":"0cb1bb15-2b83-4698-801c-6f974c940753","Type":"ContainerDied","Data":"a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5"} Jul 07 00:16:41.475324 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.475074 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" Jul 07 00:16:41.475324 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.475117 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k" event={"ID":"0cb1bb15-2b83-4698-801c-6f974c940753","Type":"ContainerDied","Data":"0ce97ee01a553a8ab302fa07f89c468940cc7e9651b64d5abf94ca28d00639d0"} Jul 07 00:16:41.475324 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.475147 2558 scope.go:117] "RemoveContainer" containerID="a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5" Jul 07 00:16:41.484168 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.484148 2558 scope.go:117] "RemoveContainer" containerID="a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5" Jul 07 00:16:41.484484 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:16:41.484463 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5\": container with ID starting with a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5 not found: ID does not exist" containerID="a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5" Jul 07 00:16:41.484534 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.484494 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5"} err="failed to get container status \"a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5\": rpc error: code = NotFound desc = could not find container \"a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5\": container with ID starting with a169ea02b073bce2ba100114ce455cb409c297c8d93a94d8c523b24ab5d17da5 not found: ID does not exist" Jul 07 00:16:41.497625 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.497594 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k"] Jul 07 00:16:41.499210 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:41.499186 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-104fd30e-57d5975bd9-stc4k"] Jul 07 00:16:42.051806 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:42.051773 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cb1bb15-2b83-4698-801c-6f974c940753" path="/var/lib/kubelet/pods/0cb1bb15-2b83-4698-801c-6f974c940753/volumes" Jul 07 00:16:46.336353 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:46.336313 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:51.394373 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:16:51.394339 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:28.145614 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:28.145579 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:28.594713 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:28.594679 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:34.207673 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:34.207626 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:48.379705 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:48.379666 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:48.521400 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:48.521362 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:48.634531 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:48.634440 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p"] Jul 07 00:17:48.634788 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:48.634753 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" containerID="cri-o://20ee59b74b923d38890af39995fd24dfdf33f77a5418c18c4f6f0a6b01aeb1b9" gracePeriod=30 Jul 07 00:17:48.651410 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:48.651377 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:48.879479 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:48.879444 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:49.916292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:49.916263 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:50.011294 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:50.011259 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:50.110868 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:50.110824 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:50.835871 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:50.835806 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:50.931325 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:50.931291 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:51.031439 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:51.031406 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:51.564297 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:51.564266 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:52.414489 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:52.414422 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:53.804159 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:53.804128 2558 generic.go:356] "Generic (PLEG): container finished" podID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerID="20ee59b74b923d38890af39995fd24dfdf33f77a5418c18c4f6f0a6b01aeb1b9" exitCode=0 Jul 07 00:17:53.804505 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:53.804173 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" event={"ID":"76d3d593-5fbe-468a-98e2-e1e39fa382f7","Type":"ContainerDied","Data":"20ee59b74b923d38890af39995fd24dfdf33f77a5418c18c4f6f0a6b01aeb1b9"} Jul 07 00:17:53.804505 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:53.804229 2558 scope.go:117] "RemoveContainer" containerID="58d4c1f11ff5391918f081edb95d8c6c8535dfdc81997b5da47a3e0395445440" Jul 07 00:17:53.890531 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:53.890501 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:17:54.022658 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.022621 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-certs\") pod \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " Jul 07 00:17:54.022658 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.022657 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-data\") pod \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " Jul 07 00:17:54.022961 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.022680 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-socket\") pod \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " Jul 07 00:17:54.022961 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.022767 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-token\") pod \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " Jul 07 00:17:54.022961 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.022805 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-envoy\") pod \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " Jul 07 00:17:54.022961 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.022837 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-credential-socket\") pod \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " Jul 07 00:17:54.022961 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.022889 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istiod-ca-cert\") pod \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " Jul 07 00:17:54.022961 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.022919 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgvlf\" (UniqueName: \"kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-kube-api-access-mgvlf\") pod \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " Jul 07 00:17:54.022961 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.022953 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-podinfo\") pod \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\" (UID: \"76d3d593-5fbe-468a-98e2-e1e39fa382f7\") " Jul 07 00:17:54.023332 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.022991 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76d3d593-5fbe-468a-98e2-e1e39fa382f7" (UID: "76d3d593-5fbe-468a-98e2-e1e39fa382f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:17:54.023332 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.023147 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76d3d593-5fbe-468a-98e2-e1e39fa382f7" (UID: "76d3d593-5fbe-468a-98e2-e1e39fa382f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:17:54.023332 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.023157 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76d3d593-5fbe-468a-98e2-e1e39fa382f7" (UID: "76d3d593-5fbe-468a-98e2-e1e39fa382f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:17:54.023332 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.023288 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-credential-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:17:54.023332 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.023308 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-certs\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:17:54.023332 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.023322 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-workload-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:17:54.023332 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.023319 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76d3d593-5fbe-468a-98e2-e1e39fa382f7" (UID: "76d3d593-5fbe-468a-98e2-e1e39fa382f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:17:54.023886 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.023624 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "76d3d593-5fbe-468a-98e2-e1e39fa382f7" (UID: "76d3d593-5fbe-468a-98e2-e1e39fa382f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:17:54.025250 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.025222 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76d3d593-5fbe-468a-98e2-e1e39fa382f7" (UID: "76d3d593-5fbe-468a-98e2-e1e39fa382f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:17:54.025250 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.025236 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-kube-api-access-mgvlf" (OuterVolumeSpecName: "kube-api-access-mgvlf") pod "76d3d593-5fbe-468a-98e2-e1e39fa382f7" (UID: "76d3d593-5fbe-468a-98e2-e1e39fa382f7"). InnerVolumeSpecName "kube-api-access-mgvlf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:17:54.025390 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.025298 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76d3d593-5fbe-468a-98e2-e1e39fa382f7" (UID: "76d3d593-5fbe-468a-98e2-e1e39fa382f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:17:54.025390 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.025332 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "76d3d593-5fbe-468a-98e2-e1e39fa382f7" (UID: "76d3d593-5fbe-468a-98e2-e1e39fa382f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:17:54.124491 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.124455 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-data\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:17:54.124491 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.124487 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-token\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:17:54.124491 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.124496 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-envoy\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:17:54.124491 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.124504 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istiod-ca-cert\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:17:54.124774 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.124513 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgvlf\" (UniqueName: \"kubernetes.io/projected/76d3d593-5fbe-468a-98e2-e1e39fa382f7-kube-api-access-mgvlf\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:17:54.124774 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.124528 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76d3d593-5fbe-468a-98e2-e1e39fa382f7-istio-podinfo\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:17:54.809907 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.809877 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" Jul 07 00:17:54.810385 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.809871 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p" event={"ID":"76d3d593-5fbe-468a-98e2-e1e39fa382f7","Type":"ContainerDied","Data":"31f586776a28d0f3d113ecbb672a88768ca3e281843a0e165f0aeea3ca99ae8f"} Jul 07 00:17:54.810385 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.810004 2558 scope.go:117] "RemoveContainer" containerID="20ee59b74b923d38890af39995fd24dfdf33f77a5418c18c4f6f0a6b01aeb1b9" Jul 07 00:17:54.828452 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.828418 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p"] Jul 07 00:17:54.831634 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:54.831601 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-104fd30e-openshift-default-55df8769f-b9x6p"] Jul 07 00:17:56.051214 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:56.051184 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" path="/var/lib/kubelet/pods/76d3d593-5fbe-468a-98e2-e1e39fa382f7/volumes" Jul 07 00:17:58.073691 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:58.073655 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:58.170470 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:58.170436 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:17:58.720364 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:17:58.720331 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:04.640237 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:04.640197 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:04.746150 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:04.746115 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:04.840694 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:04.840662 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:15.327433 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:15.327393 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:15.675353 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:15.675318 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:36.653359 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:36.653274 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:40.075020 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:40.074986 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:18:40.078914 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:40.078888 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:18:42.620170 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:42.620138 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:42.687832 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:42.687795 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:43.005231 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.005199 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:43.291144 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.291061 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9"] Jul 07 00:18:43.291553 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.291538 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" Jul 07 00:18:43.291596 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.291557 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" Jul 07 00:18:43.291596 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.291571 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb1bb15-2b83-4698-801c-6f974c940753" containerName="maas-api" Jul 07 00:18:43.291596 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.291580 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb1bb15-2b83-4698-801c-6f974c940753" containerName="maas-api" Jul 07 00:18:43.291596 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.291591 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" Jul 07 00:18:43.291733 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.291601 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" Jul 07 00:18:43.291733 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.291685 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" Jul 07 00:18:43.291733 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.291696 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d3d593-5fbe-468a-98e2-e1e39fa382f7" containerName="istio-proxy" Jul 07 00:18:43.291733 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.291706 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb1bb15-2b83-4698-801c-6f974c940753" containerName="maas-api" Jul 07 00:18:43.295038 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.295018 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.297622 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.297598 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-474e44-openshift-default-dockercfg-vktq6\"" Jul 07 00:18:43.306118 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.306091 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9"] Jul 07 00:18:43.312950 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.312923 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:43.384650 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.384611 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:43.450161 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.450108 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-token\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.450161 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.450162 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-credential-socket\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.450397 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.450197 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-envoy\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.450397 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.450233 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-socket\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.450397 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.450266 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8rct\" (UniqueName: \"kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-kube-api-access-f8rct\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.450397 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.450292 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-data\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.450397 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.450307 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-podinfo\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.450560 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.450391 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-certs\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.450560 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.450437 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istiod-ca-cert\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.551902 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.551764 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-credential-socket\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.551902 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.551813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-envoy\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.551902 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.551872 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-socket\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.551902 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.551900 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8rct\" (UniqueName: \"kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-kube-api-access-f8rct\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.552241 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.551932 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-data\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.552241 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.552068 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-podinfo\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.552241 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.552156 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-certs\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.552241 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.552186 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istiod-ca-cert\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.552458 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.552258 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-credential-socket\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.552458 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.552263 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-token\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.552458 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.552348 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-socket\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.552624 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.552526 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-certs\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.552733 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.552713 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-data\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.552874 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.552773 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istiod-ca-cert\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.554400 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.554377 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-envoy\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.554749 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.554730 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-podinfo\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.560555 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.560528 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8rct\" (UniqueName: \"kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-kube-api-access-f8rct\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.562300 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.562276 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-token\") pod \"e2e-inf-a-474e44-openshift-default-57b846748f-sppz9\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.606434 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.606397 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:43.747001 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.746915 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9"] Jul 07 00:18:43.749570 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:18:43.749540 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934f6b7a_d3ac_45b8_a6f9_22a4f8efe97c.slice/crio-09a20241c929e327cb5bb8b501e130da6f211bb3331c5d34c44be4cb46d428d6 WatchSource:0}: Error finding container 09a20241c929e327cb5bb8b501e130da6f211bb3331c5d34c44be4cb46d428d6: Status 404 returned error can't find the container with id 09a20241c929e327cb5bb8b501e130da6f211bb3331c5d34c44be4cb46d428d6 Jul 07 00:18:43.751800 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.751767 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:18:43.751899 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.751839 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:18:43.751899 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:43.751892 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:18:44.052373 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:44.052324 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" event={"ID":"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c","Type":"ContainerStarted","Data":"f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6"} Jul 07 00:18:44.052373 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:44.052381 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" event={"ID":"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c","Type":"ContainerStarted","Data":"09a20241c929e327cb5bb8b501e130da6f211bb3331c5d34c44be4cb46d428d6"} Jul 07 00:18:44.068427 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:44.068315 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" podStartSLOduration=1.068297815 podStartE2EDuration="1.068297815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:18:44.065547935 +0000 UTC m=+1804.575708892" watchObservedRunningTime="2026-07-07 00:18:44.068297815 +0000 UTC m=+1804.578458771" Jul 07 00:18:44.606836 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:44.606796 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:44.611997 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:44.611964 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:45.049971 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:45.049936 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:45.050931 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:45.050913 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:18:46.141181 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.141142 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:46.198379 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.198347 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f"] Jul 07 00:18:46.201886 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.201840 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.204600 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.204576 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-474e44-serving-cert\"" Jul 07 00:18:46.213638 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.213583 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f"] Jul 07 00:18:46.231122 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.231082 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:46.276096 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.276049 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbh7h\" (UniqueName: \"kubernetes.io/projected/f048cebd-ede7-4d13-a99d-f6de55597c3f-kube-api-access-mbh7h\") pod \"maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.276291 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.276106 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f048cebd-ede7-4d13-a99d-f6de55597c3f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.276291 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.276223 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f048cebd-ede7-4d13-a99d-f6de55597c3f-maas-api-tls\") pod \"maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.301063 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.301014 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:46.377535 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.377493 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f048cebd-ede7-4d13-a99d-f6de55597c3f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.377747 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.377570 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f048cebd-ede7-4d13-a99d-f6de55597c3f-maas-api-tls\") pod \"maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.377747 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.377619 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbh7h\" (UniqueName: \"kubernetes.io/projected/f048cebd-ede7-4d13-a99d-f6de55597c3f-kube-api-access-mbh7h\") pod \"maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.378269 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.378243 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f048cebd-ede7-4d13-a99d-f6de55597c3f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.380216 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.380191 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f048cebd-ede7-4d13-a99d-f6de55597c3f-maas-api-tls\") pod \"maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.387883 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.387840 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbh7h\" (UniqueName: \"kubernetes.io/projected/f048cebd-ede7-4d13-a99d-f6de55597c3f-kube-api-access-mbh7h\") pod \"maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.429721 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.429680 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:46.514794 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.514756 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:46.655052 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.654913 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f"] Jul 07 00:18:46.657980 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:18:46.657949 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf048cebd_ede7_4d13_a99d_f6de55597c3f.slice/crio-29f39f437d6da7ea4b7d15d12fc328d9ab2659eba3ad30fcc738191a429e4bcf WatchSource:0}: Error finding container 29f39f437d6da7ea4b7d15d12fc328d9ab2659eba3ad30fcc738191a429e4bcf: Status 404 returned error can't find the container with id 29f39f437d6da7ea4b7d15d12fc328d9ab2659eba3ad30fcc738191a429e4bcf Jul 07 00:18:46.659619 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.659599 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:18:46.766335 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.766248 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:46.867763 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:46.867734 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:47.059669 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:47.059573 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" event={"ID":"f048cebd-ede7-4d13-a99d-f6de55597c3f","Type":"ContainerStarted","Data":"29f39f437d6da7ea4b7d15d12fc328d9ab2659eba3ad30fcc738191a429e4bcf"} Jul 07 00:18:48.071765 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:48.071725 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" event={"ID":"f048cebd-ede7-4d13-a99d-f6de55597c3f","Type":"ContainerStarted","Data":"bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4"} Jul 07 00:18:48.072240 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:48.071951 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:48.089070 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:48.089019 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" podStartSLOduration=1.478128899 podStartE2EDuration="2.089001608s" totalImagesPullingTime="610.872709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:18:48.087604114 +0000 UTC m=+1808.597765070" watchObservedRunningTime="2026-07-07 00:18:48.089001608 +0000 UTC m=+1808.599162565" Jul 07 00:18:50.546987 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:50.546951 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:50.897884 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:50.897776 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:51.215023 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:51.214990 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:51.668406 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:51.668326 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:54.082243 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.082216 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:18:54.668376 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.668341 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:54.764606 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.764563 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz"] Jul 07 00:18:54.768301 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.768273 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:54.771122 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.771098 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-ea7465-serving-cert\"" Jul 07 00:18:54.790086 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.790047 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz"] Jul 07 00:18:54.823246 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.823207 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:54.852593 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.852555 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b039f091-11a3-4719-9e6a-93de562f97f8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ea7465-685657c795-x4xgz\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:54.852793 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.852616 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b039f091-11a3-4719-9e6a-93de562f97f8-maas-api-tls\") pod \"maas-api-e2e-inf-b-ea7465-685657c795-x4xgz\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:54.852793 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.852677 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvngd\" (UniqueName: \"kubernetes.io/projected/b039f091-11a3-4719-9e6a-93de562f97f8-kube-api-access-fvngd\") pod \"maas-api-e2e-inf-b-ea7465-685657c795-x4xgz\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:54.921546 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.921519 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:54.953790 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.953749 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b039f091-11a3-4719-9e6a-93de562f97f8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ea7465-685657c795-x4xgz\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:54.953989 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.953822 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b039f091-11a3-4719-9e6a-93de562f97f8-maas-api-tls\") pod \"maas-api-e2e-inf-b-ea7465-685657c795-x4xgz\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:54.953989 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.953920 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvngd\" (UniqueName: \"kubernetes.io/projected/b039f091-11a3-4719-9e6a-93de562f97f8-kube-api-access-fvngd\") pod \"maas-api-e2e-inf-b-ea7465-685657c795-x4xgz\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:54.954086 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:18:54.953988 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-ea7465-serving-cert: secret "maas-api-e2e-inf-b-ea7465-serving-cert" not found Jul 07 00:18:54.954086 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:18:54.954068 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b039f091-11a3-4719-9e6a-93de562f97f8-maas-api-tls podName:b039f091-11a3-4719-9e6a-93de562f97f8 nodeName:}" failed. No retries permitted until 2026-07-07 00:18:55.454047981 +0000 UTC m=+1815.964208916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b039f091-11a3-4719-9e6a-93de562f97f8-maas-api-tls") pod "maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" (UID: "b039f091-11a3-4719-9e6a-93de562f97f8") : secret "maas-api-e2e-inf-b-ea7465-serving-cert" not found Jul 07 00:18:54.954541 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.954522 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b039f091-11a3-4719-9e6a-93de562f97f8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ea7465-685657c795-x4xgz\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:54.963367 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:54.963342 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvngd\" (UniqueName: \"kubernetes.io/projected/b039f091-11a3-4719-9e6a-93de562f97f8-kube-api-access-fvngd\") pod \"maas-api-e2e-inf-b-ea7465-685657c795-x4xgz\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:55.150869 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:55.150820 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:55.306569 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:55.306470 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:55.404067 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:55.404030 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:55.458172 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:55.458132 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b039f091-11a3-4719-9e6a-93de562f97f8-maas-api-tls\") pod \"maas-api-e2e-inf-b-ea7465-685657c795-x4xgz\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:55.460863 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:55.460832 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b039f091-11a3-4719-9e6a-93de562f97f8-maas-api-tls\") pod \"maas-api-e2e-inf-b-ea7465-685657c795-x4xgz\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:55.681740 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:55.681693 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:55.866311 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:55.866284 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz"] Jul 07 00:18:55.868696 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:18:55.868663 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb039f091_11a3_4719_9e6a_93de562f97f8.slice/crio-d2d0fb2118d02858eb995c6ddad371febd36342bf5711056695036b8114c953e WatchSource:0}: Error finding container d2d0fb2118d02858eb995c6ddad371febd36342bf5711056695036b8114c953e: Status 404 returned error can't find the container with id d2d0fb2118d02858eb995c6ddad371febd36342bf5711056695036b8114c953e Jul 07 00:18:56.112378 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:56.112284 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" event={"ID":"b039f091-11a3-4719-9e6a-93de562f97f8","Type":"ContainerStarted","Data":"d2d0fb2118d02858eb995c6ddad371febd36342bf5711056695036b8114c953e"} Jul 07 00:18:57.117952 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:57.117917 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" event={"ID":"b039f091-11a3-4719-9e6a-93de562f97f8","Type":"ContainerStarted","Data":"399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f"} Jul 07 00:18:57.118355 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:57.118000 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:18:57.139521 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:57.139472 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" podStartSLOduration=2.725153223 podStartE2EDuration="3.139455371s" totalImagesPullingTime="414.302148ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:18:57.137537719 +0000 UTC m=+1817.647698689" watchObservedRunningTime="2026-07-07 00:18:57.139455371 +0000 UTC m=+1817.649616327" Jul 07 00:18:59.178405 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:18:59.178371 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:19:01.091042 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:01.090999 2558 patch_prober.go:28] interesting pod/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 00:19:01.091463 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:01.091072 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 00:19:02.143354 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:02.143320 2558 generic.go:356] "Generic (PLEG): container finished" podID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerID="f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6" exitCode=0 Jul 07 00:19:02.143918 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:02.143405 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" event={"ID":"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c","Type":"ContainerDied","Data":"f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6"} Jul 07 00:19:02.143995 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:02.143962 2558 scope.go:117] "RemoveContainer" containerID="f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6" Jul 07 00:19:02.145000 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:02.144971 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:19:02.145072 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:02.145044 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:19:02.145114 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:02.145098 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:19:03.128268 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:03.128240 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:19:03.150655 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:03.150621 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" event={"ID":"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c","Type":"ContainerStarted","Data":"ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526"} Jul 07 00:19:03.607552 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:03.607512 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:19:03.607793 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:03.607768 2558 patch_prober.go:28] interesting pod/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 07 00:19:03.607841 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:03.607823 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 07 00:19:04.606942 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:04.606907 2558 patch_prober.go:28] interesting pod/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 07 00:19:04.607360 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:04.606969 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 07 00:19:05.607842 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:05.607793 2558 patch_prober.go:28] interesting pod/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 07 00:19:05.608264 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:05.607908 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 07 00:19:06.607623 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:06.607579 2558 patch_prober.go:28] interesting pod/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 07 00:19:06.607818 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:06.607660 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 07 00:19:07.468426 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:07.468393 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:19:07.569652 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:07.569616 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:19:07.646817 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:07.646787 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:19:07.647161 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:07.647141 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:19:07.647992 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:07.647972 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:19:16.942139 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:16.942105 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:19:17.044064 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:17.044023 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:19:17.918021 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:17.917980 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:19:43.513141 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:19:43.513100 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:02.910617 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:02.910526 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:03.802150 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:03.802107 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:04.064655 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:04.064573 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:04.395126 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:04.395043 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:04.496197 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:04.496149 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:04.607706 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:04.607666 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:04.782128 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:04.782093 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f"] Jul 07 00:20:04.782423 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:04.782383 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" podUID="f048cebd-ede7-4d13-a99d-f6de55597c3f" containerName="maas-api" containerID="cri-o://bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4" gracePeriod=30 Jul 07 00:20:04.810746 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:04.810711 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:05.043970 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.043902 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:20:05.089709 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.089669 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbh7h\" (UniqueName: \"kubernetes.io/projected/f048cebd-ede7-4d13-a99d-f6de55597c3f-kube-api-access-mbh7h\") pod \"f048cebd-ede7-4d13-a99d-f6de55597c3f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " Jul 07 00:20:05.090216 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.089761 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f048cebd-ede7-4d13-a99d-f6de55597c3f-openshift-service-ca\") pod \"f048cebd-ede7-4d13-a99d-f6de55597c3f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " Jul 07 00:20:05.090216 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.089794 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f048cebd-ede7-4d13-a99d-f6de55597c3f-maas-api-tls\") pod \"f048cebd-ede7-4d13-a99d-f6de55597c3f\" (UID: \"f048cebd-ede7-4d13-a99d-f6de55597c3f\") " Jul 07 00:20:05.090315 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.090218 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f048cebd-ede7-4d13-a99d-f6de55597c3f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f048cebd-ede7-4d13-a99d-f6de55597c3f" (UID: "f048cebd-ede7-4d13-a99d-f6de55597c3f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:20:05.092094 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.092064 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f048cebd-ede7-4d13-a99d-f6de55597c3f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f048cebd-ede7-4d13-a99d-f6de55597c3f" (UID: "f048cebd-ede7-4d13-a99d-f6de55597c3f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:20:05.092203 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.092098 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f048cebd-ede7-4d13-a99d-f6de55597c3f-kube-api-access-mbh7h" (OuterVolumeSpecName: "kube-api-access-mbh7h") pod "f048cebd-ede7-4d13-a99d-f6de55597c3f" (UID: "f048cebd-ede7-4d13-a99d-f6de55597c3f"). InnerVolumeSpecName "kube-api-access-mbh7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:05.190551 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.190500 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f048cebd-ede7-4d13-a99d-f6de55597c3f-maas-api-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:05.190551 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.190545 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbh7h\" (UniqueName: \"kubernetes.io/projected/f048cebd-ede7-4d13-a99d-f6de55597c3f-kube-api-access-mbh7h\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:05.190551 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.190556 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f048cebd-ede7-4d13-a99d-f6de55597c3f-openshift-service-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:05.437176 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.437138 2558 generic.go:356] "Generic (PLEG): container finished" podID="f048cebd-ede7-4d13-a99d-f6de55597c3f" containerID="bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4" exitCode=0 Jul 07 00:20:05.437360 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.437186 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" Jul 07 00:20:05.437360 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.437221 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" event={"ID":"f048cebd-ede7-4d13-a99d-f6de55597c3f","Type":"ContainerDied","Data":"bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4"} Jul 07 00:20:05.437360 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.437274 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f" event={"ID":"f048cebd-ede7-4d13-a99d-f6de55597c3f","Type":"ContainerDied","Data":"29f39f437d6da7ea4b7d15d12fc328d9ab2659eba3ad30fcc738191a429e4bcf"} Jul 07 00:20:05.437360 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.437297 2558 scope.go:117] "RemoveContainer" containerID="bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4" Jul 07 00:20:05.446727 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.446704 2558 scope.go:117] "RemoveContainer" containerID="bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4" Jul 07 00:20:05.447133 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:20:05.447113 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4\": container with ID starting with bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4 not found: ID does not exist" containerID="bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4" Jul 07 00:20:05.447222 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.447142 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4"} err="failed to get container status \"bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4\": rpc error: code = NotFound desc = could not find container \"bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4\": container with ID starting with bf8306b64263eb9aa4708fde44b270ce22765b4ac127740b1db80e3b041874f4 not found: ID does not exist" Jul 07 00:20:05.458773 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.458739 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f"] Jul 07 00:20:05.462973 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:05.462948 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-474e44-766fd5548d-rnk9f"] Jul 07 00:20:06.051399 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:06.051367 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f048cebd-ede7-4d13-a99d-f6de55597c3f" path="/var/lib/kubelet/pods/f048cebd-ede7-4d13-a99d-f6de55597c3f/volumes" Jul 07 00:20:09.475055 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:09.475015 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:12.233884 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:12.233830 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:12.290471 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:12.290440 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:22.938458 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:22.938421 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:23.038000 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:23.037960 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:46.126149 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:46.126101 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:46.366251 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:46.366219 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9"] Jul 07 00:20:46.366540 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:46.366512 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" containerID="cri-o://ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526" gracePeriod=30 Jul 07 00:20:46.382356 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:46.382245 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:46.625752 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:46.625716 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:46.788540 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:46.788504 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:47.206181 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.206123 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:47.321789 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.321737 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:47.422817 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.422757 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:47.542317 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.542226 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz"] Jul 07 00:20:47.542521 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.542485 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" podUID="b039f091-11a3-4719-9e6a-93de562f97f8" containerName="maas-api" containerID="cri-o://399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f" gracePeriod=30 Jul 07 00:20:47.567038 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.567012 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:47.791941 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.791909 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:20:47.874275 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.874184 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvngd\" (UniqueName: \"kubernetes.io/projected/b039f091-11a3-4719-9e6a-93de562f97f8-kube-api-access-fvngd\") pod \"b039f091-11a3-4719-9e6a-93de562f97f8\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " Jul 07 00:20:47.874275 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.874245 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b039f091-11a3-4719-9e6a-93de562f97f8-openshift-service-ca\") pod \"b039f091-11a3-4719-9e6a-93de562f97f8\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " Jul 07 00:20:47.874464 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.874276 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b039f091-11a3-4719-9e6a-93de562f97f8-maas-api-tls\") pod \"b039f091-11a3-4719-9e6a-93de562f97f8\" (UID: \"b039f091-11a3-4719-9e6a-93de562f97f8\") " Jul 07 00:20:47.874738 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.874712 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b039f091-11a3-4719-9e6a-93de562f97f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b039f091-11a3-4719-9e6a-93de562f97f8" (UID: "b039f091-11a3-4719-9e6a-93de562f97f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:20:47.876516 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.876486 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b039f091-11a3-4719-9e6a-93de562f97f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b039f091-11a3-4719-9e6a-93de562f97f8" (UID: "b039f091-11a3-4719-9e6a-93de562f97f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:20:47.876623 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.876566 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b039f091-11a3-4719-9e6a-93de562f97f8-kube-api-access-fvngd" (OuterVolumeSpecName: "kube-api-access-fvngd") pod "b039f091-11a3-4719-9e6a-93de562f97f8" (UID: "b039f091-11a3-4719-9e6a-93de562f97f8"). InnerVolumeSpecName "kube-api-access-fvngd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:47.974974 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.974938 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b039f091-11a3-4719-9e6a-93de562f97f8-openshift-service-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:47.974974 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.974971 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b039f091-11a3-4719-9e6a-93de562f97f8-maas-api-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:47.974974 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:47.974982 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvngd\" (UniqueName: \"kubernetes.io/projected/b039f091-11a3-4719-9e6a-93de562f97f8-kube-api-access-fvngd\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:48.641875 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:48.641816 2558 generic.go:356] "Generic (PLEG): container finished" podID="b039f091-11a3-4719-9e6a-93de562f97f8" containerID="399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f" exitCode=0 Jul 07 00:20:48.642285 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:48.641889 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" Jul 07 00:20:48.642285 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:48.641900 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" event={"ID":"b039f091-11a3-4719-9e6a-93de562f97f8","Type":"ContainerDied","Data":"399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f"} Jul 07 00:20:48.642285 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:48.641960 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz" event={"ID":"b039f091-11a3-4719-9e6a-93de562f97f8","Type":"ContainerDied","Data":"d2d0fb2118d02858eb995c6ddad371febd36342bf5711056695036b8114c953e"} Jul 07 00:20:48.642285 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:48.641984 2558 scope.go:117] "RemoveContainer" containerID="399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f" Jul 07 00:20:48.650982 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:48.650962 2558 scope.go:117] "RemoveContainer" containerID="399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f" Jul 07 00:20:48.651257 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:20:48.651234 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f\": container with ID starting with 399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f not found: ID does not exist" containerID="399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f" Jul 07 00:20:48.651295 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:48.651270 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f"} err="failed to get container status \"399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f\": rpc error: code = NotFound desc = could not find container \"399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f\": container with ID starting with 399c838162db6aeac1b2b50046fa046e241c094f49118a9d9fb18d78d7f1887f not found: ID does not exist" Jul 07 00:20:48.658824 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:48.658792 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz"] Jul 07 00:20:48.662553 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:48.662528 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ea7465-685657c795-x4xgz"] Jul 07 00:20:50.052880 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:50.052832 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b039f091-11a3-4719-9e6a-93de562f97f8" path="/var/lib/kubelet/pods/b039f091-11a3-4719-9e6a-93de562f97f8/volumes" Jul 07 00:20:51.619000 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.618966 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:20:51.658330 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.658247 2558 generic.go:356] "Generic (PLEG): container finished" podID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerID="ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526" exitCode=0 Jul 07 00:20:51.658330 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.658308 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" Jul 07 00:20:51.658505 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.658337 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" event={"ID":"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c","Type":"ContainerDied","Data":"ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526"} Jul 07 00:20:51.658505 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.658419 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9" event={"ID":"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c","Type":"ContainerDied","Data":"09a20241c929e327cb5bb8b501e130da6f211bb3331c5d34c44be4cb46d428d6"} Jul 07 00:20:51.658505 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.658442 2558 scope.go:117] "RemoveContainer" containerID="ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526" Jul 07 00:20:51.667098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.667078 2558 scope.go:117] "RemoveContainer" containerID="f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6" Jul 07 00:20:51.675046 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.675023 2558 scope.go:117] "RemoveContainer" containerID="ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526" Jul 07 00:20:51.675338 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:20:51.675317 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526\": container with ID starting with ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526 not found: ID does not exist" containerID="ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526" Jul 07 00:20:51.675405 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.675348 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526"} err="failed to get container status \"ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526\": rpc error: code = NotFound desc = could not find container \"ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526\": container with ID starting with ea753d56aaa145520e49c02988302acf8868c4b8331e3e0cd67ff8b86a646526 not found: ID does not exist" Jul 07 00:20:51.675405 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.675366 2558 scope.go:117] "RemoveContainer" containerID="f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6" Jul 07 00:20:51.675604 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:20:51.675591 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6\": container with ID starting with f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6 not found: ID does not exist" containerID="f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6" Jul 07 00:20:51.675646 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.675608 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6"} err="failed to get container status \"f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6\": rpc error: code = NotFound desc = could not find container \"f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6\": container with ID starting with f8ad53967a22a85adb24ee704a546426203f9d989608edfb84f89735c6ab0be6 not found: ID does not exist" Jul 07 00:20:51.699035 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.698982 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8rct\" (UniqueName: \"kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-kube-api-access-f8rct\") pod \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " Jul 07 00:20:51.699035 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699045 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-data\") pod \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " Jul 07 00:20:51.699292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699064 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istiod-ca-cert\") pod \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " Jul 07 00:20:51.699292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699086 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-token\") pod \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " Jul 07 00:20:51.699292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699119 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-certs\") pod \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " Jul 07 00:20:51.699292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699150 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-credential-socket\") pod \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " Jul 07 00:20:51.699292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699174 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-podinfo\") pod \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " Jul 07 00:20:51.699292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699237 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-envoy\") pod \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " Jul 07 00:20:51.699292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699269 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-socket\") pod \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\" (UID: \"934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c\") " Jul 07 00:20:51.699641 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699440 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" (UID: "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:51.699641 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699461 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" (UID: "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:51.699641 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699613 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-credential-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:51.699641 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699632 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-certs\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:51.699950 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699642 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" (UID: "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:51.699950 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.699797 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" (UID: "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:20:51.700378 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.700350 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-data" (OuterVolumeSpecName: "istio-data") pod "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" (UID: "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:51.701453 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.701426 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-token" (OuterVolumeSpecName: "istio-token") pod "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" (UID: "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:51.701453 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.701447 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-kube-api-access-f8rct" (OuterVolumeSpecName: "kube-api-access-f8rct") pod "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" (UID: "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c"). InnerVolumeSpecName "kube-api-access-f8rct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:51.701925 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.701903 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" (UID: "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:20:51.701925 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.701910 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" (UID: "934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:51.800091 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.800040 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-envoy\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:51.800091 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.800083 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-workload-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:51.800091 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.800094 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8rct\" (UniqueName: \"kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-kube-api-access-f8rct\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:51.800091 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.800104 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-data\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:51.800091 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.800113 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istiod-ca-cert\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:51.800406 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.800120 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-token\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:51.800406 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.800127 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c-istio-podinfo\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:20:51.980965 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.980930 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9"] Jul 07 00:20:51.984481 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:51.984454 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-474e44-openshift-default-57b846748f-sppz9"] Jul 07 00:20:52.051866 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:52.051819 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" path="/var/lib/kubelet/pods/934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c/volumes" Jul 07 00:20:53.143046 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:53.143004 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:53.211639 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:53.211591 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:53.477369 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:53.477332 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:53.681091 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:53.681057 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:53.998879 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:53.998831 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:54.284649 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.284563 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x"] Jul 07 00:20:54.285059 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.284895 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" Jul 07 00:20:54.285059 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.284907 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" Jul 07 00:20:54.285059 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.284921 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b039f091-11a3-4719-9e6a-93de562f97f8" containerName="maas-api" Jul 07 00:20:54.285059 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.284926 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b039f091-11a3-4719-9e6a-93de562f97f8" containerName="maas-api" Jul 07 00:20:54.285059 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.284934 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f048cebd-ede7-4d13-a99d-f6de55597c3f" containerName="maas-api" Jul 07 00:20:54.285059 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.284940 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f048cebd-ede7-4d13-a99d-f6de55597c3f" containerName="maas-api" Jul 07 00:20:54.285059 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.284998 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b039f091-11a3-4719-9e6a-93de562f97f8" containerName="maas-api" Jul 07 00:20:54.285059 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.285006 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" Jul 07 00:20:54.285059 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.285012 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f048cebd-ede7-4d13-a99d-f6de55597c3f" containerName="maas-api" Jul 07 00:20:54.289589 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.289562 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.292315 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.292294 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-91d197-openshift-default-dockercfg-qldwj\"" Jul 07 00:20:54.300967 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.300941 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x"] Jul 07 00:20:54.312323 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.312298 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:54.321692 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.321660 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-certs\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.321860 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.321695 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-envoy\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.321860 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.321728 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-token\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.321860 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.321814 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qgb6\" (UniqueName: \"kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-kube-api-access-8qgb6\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.322014 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.321870 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-data\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.322014 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.321896 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-socket\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.322014 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.321922 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56364ead-957f-4aa3-b0b4-516a18f917b3-istiod-ca-cert\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.322014 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.321989 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-podinfo\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.322170 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.322017 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-credential-socket\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423058 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423015 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-data\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423058 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423058 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-socket\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423327 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423087 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56364ead-957f-4aa3-b0b4-516a18f917b3-istiod-ca-cert\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423327 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423133 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-podinfo\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423327 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423160 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-credential-socket\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423327 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423190 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-certs\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423327 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423217 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-envoy\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423591 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423323 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-token\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423591 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423407 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qgb6\" (UniqueName: \"kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-kube-api-access-8qgb6\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423591 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423511 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-socket\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423825 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423601 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-credential-socket\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423825 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423623 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-certs\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.423825 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423779 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-data\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.424136 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.423917 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56364ead-957f-4aa3-b0b4-516a18f917b3-istiod-ca-cert\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.425782 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.425761 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-envoy\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.425932 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.425916 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-podinfo\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.434908 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.434843 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-token\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.434908 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.434905 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qgb6\" (UniqueName: \"kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-kube-api-access-8qgb6\") pod \"e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.601228 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.601133 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:54.660860 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.660816 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:54.745056 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.745013 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x"] Jul 07 00:20:54.747417 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:20:54.747384 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56364ead_957f_4aa3_b0b4_516a18f917b3.slice/crio-cfcd7e9c8de1a850ed6e5ec33cdf7b381882d1d3718fc275ce3cf8c6abf0f52e WatchSource:0}: Error finding container cfcd7e9c8de1a850ed6e5ec33cdf7b381882d1d3718fc275ce3cf8c6abf0f52e: Status 404 returned error can't find the container with id cfcd7e9c8de1a850ed6e5ec33cdf7b381882d1d3718fc275ce3cf8c6abf0f52e Jul 07 00:20:54.749674 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.749628 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:20:54.749776 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.749707 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:20:54.749776 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:54.749746 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:20:55.678837 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:55.678791 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" event={"ID":"56364ead-957f-4aa3-b0b4-516a18f917b3","Type":"ContainerStarted","Data":"284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20"} Jul 07 00:20:55.679298 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:55.678925 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" event={"ID":"56364ead-957f-4aa3-b0b4-516a18f917b3","Type":"ContainerStarted","Data":"cfcd7e9c8de1a850ed6e5ec33cdf7b381882d1d3718fc275ce3cf8c6abf0f52e"} Jul 07 00:20:55.706667 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:55.706603 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" podStartSLOduration=1.706587482 podStartE2EDuration="1.706587482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:20:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:20:55.70349515 +0000 UTC m=+1936.213656106" watchObservedRunningTime="2026-07-07 00:20:55.706587482 +0000 UTC m=+1936.216748503" Jul 07 00:20:56.601382 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:56.601340 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:56.606396 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:56.606370 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:56.682907 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:56.682866 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:56.683966 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:56.683947 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:20:57.584134 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.584091 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:57.681840 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.681793 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:57.765213 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.765181 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws"] Jul 07 00:20:57.765566 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.765506 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" Jul 07 00:20:57.765566 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.765516 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" Jul 07 00:20:57.765638 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.765569 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="934f6b7a-d3ac-45b8-a6f9-22a4f8efe97c" containerName="istio-proxy" Jul 07 00:20:57.768615 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.768592 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:57.771281 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.771262 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-91d197-serving-cert\"" Jul 07 00:20:57.780406 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.780377 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws"] Jul 07 00:20:57.798574 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.798543 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:57.854973 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.854878 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9966cbe-974f-4429-b252-99a75813dfb6-maas-api-tls\") pod \"maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:57.854973 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.854948 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgfhd\" (UniqueName: \"kubernetes.io/projected/d9966cbe-974f-4429-b252-99a75813dfb6-kube-api-access-kgfhd\") pod \"maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:57.854973 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.854976 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9966cbe-974f-4429-b252-99a75813dfb6-openshift-service-ca\") pod \"maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:57.956412 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.956370 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9966cbe-974f-4429-b252-99a75813dfb6-maas-api-tls\") pod \"maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:57.956610 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.956465 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgfhd\" (UniqueName: \"kubernetes.io/projected/d9966cbe-974f-4429-b252-99a75813dfb6-kube-api-access-kgfhd\") pod \"maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:57.956610 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.956494 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9966cbe-974f-4429-b252-99a75813dfb6-openshift-service-ca\") pod \"maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:57.956610 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:20:57.956531 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-91d197-serving-cert: secret "maas-api-e2e-api-a-91d197-serving-cert" not found Jul 07 00:20:57.956610 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:20:57.956604 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9966cbe-974f-4429-b252-99a75813dfb6-maas-api-tls podName:d9966cbe-974f-4429-b252-99a75813dfb6 nodeName:}" failed. No retries permitted until 2026-07-07 00:20:58.45658525 +0000 UTC m=+1938.966746185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d9966cbe-974f-4429-b252-99a75813dfb6-maas-api-tls") pod "maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" (UID: "d9966cbe-974f-4429-b252-99a75813dfb6") : secret "maas-api-e2e-api-a-91d197-serving-cert" not found Jul 07 00:20:57.957198 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.957179 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9966cbe-974f-4429-b252-99a75813dfb6-openshift-service-ca\") pod \"maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:57.966264 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:57.966236 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgfhd\" (UniqueName: \"kubernetes.io/projected/d9966cbe-974f-4429-b252-99a75813dfb6-kube-api-access-kgfhd\") pod \"maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:58.063148 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:58.063110 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:58.341150 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:58.341104 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:58.442072 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:58.442040 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:58.461078 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:58.461033 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9966cbe-974f-4429-b252-99a75813dfb6-maas-api-tls\") pod \"maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:58.463930 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:58.463903 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9966cbe-974f-4429-b252-99a75813dfb6-maas-api-tls\") pod \"maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:58.680718 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:58.680668 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:58.823035 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:58.822991 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws"] Jul 07 00:20:58.826025 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:20:58.825997 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9966cbe_974f_4429_b252_99a75813dfb6.slice/crio-000e546a09e1e04a8d062c600bfaa77833951384a45f3d46bd1f96b76cc235c2 WatchSource:0}: Error finding container 000e546a09e1e04a8d062c600bfaa77833951384a45f3d46bd1f96b76cc235c2: Status 404 returned error can't find the container with id 000e546a09e1e04a8d062c600bfaa77833951384a45f3d46bd1f96b76cc235c2 Jul 07 00:20:59.697890 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:59.697844 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" event={"ID":"d9966cbe-974f-4429-b252-99a75813dfb6","Type":"ContainerStarted","Data":"42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743"} Jul 07 00:20:59.698069 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:59.697908 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" event={"ID":"d9966cbe-974f-4429-b252-99a75813dfb6","Type":"ContainerStarted","Data":"000e546a09e1e04a8d062c600bfaa77833951384a45f3d46bd1f96b76cc235c2"} Jul 07 00:20:59.698069 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:59.697950 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:20:59.716227 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:20:59.716173 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" podStartSLOduration=2.2899351660000002 podStartE2EDuration="2.716157211s" totalImagesPullingTime="426.222045ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:20:59.715062133 +0000 UTC m=+1940.225223101" watchObservedRunningTime="2026-07-07 00:20:59.716157211 +0000 UTC m=+1940.226318215" Jul 07 00:21:02.095570 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:02.095535 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:02.594410 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:02.594378 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:02.956997 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:02.956964 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:03.042213 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:03.042180 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:05.707950 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:05.707923 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:21:06.379629 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:06.379570 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:06.464825 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:06.464788 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:06.536470 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:06.536433 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:06.785035 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:06.784998 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:07.070016 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:07.069912 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:07.169575 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:07.169531 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:10.854306 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:10.854270 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:12.551374 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:12.550614 2558 patch_prober.go:28] interesting pod/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:49708->10.132.0.47:15021: read: connection reset by peer" start-of-body= Jul 07 00:21:12.551374 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:12.550681 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:49708->10.132.0.47:15021: read: connection reset by peer" Jul 07 00:21:12.764825 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:12.764740 2558 generic.go:356] "Generic (PLEG): container finished" podID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerID="284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20" exitCode=0 Jul 07 00:21:12.764825 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:12.764779 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" event={"ID":"56364ead-957f-4aa3-b0b4-516a18f917b3","Type":"ContainerDied","Data":"284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20"} Jul 07 00:21:12.765219 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:12.765206 2558 scope.go:117] "RemoveContainer" containerID="284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20" Jul 07 00:21:12.766228 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:12.766198 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:21:12.766288 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:12.766271 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:21:12.766327 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:12.766306 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:21:13.771205 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:13.771172 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" event={"ID":"56364ead-957f-4aa3-b0b4-516a18f917b3","Type":"ContainerStarted","Data":"6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d"} Jul 07 00:21:14.601760 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:14.601663 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:21:14.602054 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:14.602023 2558 patch_prober.go:28] interesting pod/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 00:21:14.602171 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:14.602080 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 00:21:15.602405 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:15.602360 2558 patch_prober.go:28] interesting pod/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 00:21:15.602812 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:15.602443 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 00:21:16.602134 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:16.602091 2558 patch_prober.go:28] interesting pod/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 00:21:16.602319 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:16.602167 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 00:21:17.605381 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:17.605350 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:21:17.605797 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:17.605687 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:21:17.606435 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:17.606414 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:21:18.945319 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:18.945284 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:19.196684 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.196597 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:19.298130 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.298094 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:19.399560 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.399529 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:19.426203 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.426161 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws"] Jul 07 00:21:19.426505 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.426476 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" podUID="d9966cbe-974f-4429-b252-99a75813dfb6" containerName="maas-api" containerID="cri-o://42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743" gracePeriod=30 Jul 07 00:21:19.497775 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.497693 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:19.677375 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.677350 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:21:19.757036 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.756941 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9966cbe-974f-4429-b252-99a75813dfb6-maas-api-tls\") pod \"d9966cbe-974f-4429-b252-99a75813dfb6\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " Jul 07 00:21:19.757036 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.757001 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9966cbe-974f-4429-b252-99a75813dfb6-openshift-service-ca\") pod \"d9966cbe-974f-4429-b252-99a75813dfb6\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " Jul 07 00:21:19.757244 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.757074 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgfhd\" (UniqueName: \"kubernetes.io/projected/d9966cbe-974f-4429-b252-99a75813dfb6-kube-api-access-kgfhd\") pod \"d9966cbe-974f-4429-b252-99a75813dfb6\" (UID: \"d9966cbe-974f-4429-b252-99a75813dfb6\") " Jul 07 00:21:19.757440 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.757406 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9966cbe-974f-4429-b252-99a75813dfb6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9966cbe-974f-4429-b252-99a75813dfb6" (UID: "d9966cbe-974f-4429-b252-99a75813dfb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:21:19.759288 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.759253 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9966cbe-974f-4429-b252-99a75813dfb6-kube-api-access-kgfhd" (OuterVolumeSpecName: "kube-api-access-kgfhd") pod "d9966cbe-974f-4429-b252-99a75813dfb6" (UID: "d9966cbe-974f-4429-b252-99a75813dfb6"). InnerVolumeSpecName "kube-api-access-kgfhd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:21:19.759288 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.759256 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9966cbe-974f-4429-b252-99a75813dfb6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9966cbe-974f-4429-b252-99a75813dfb6" (UID: "d9966cbe-974f-4429-b252-99a75813dfb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:21:19.802303 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.802269 2558 generic.go:356] "Generic (PLEG): container finished" podID="d9966cbe-974f-4429-b252-99a75813dfb6" containerID="42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743" exitCode=0 Jul 07 00:21:19.802489 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.802315 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" Jul 07 00:21:19.802489 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.802336 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" event={"ID":"d9966cbe-974f-4429-b252-99a75813dfb6","Type":"ContainerDied","Data":"42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743"} Jul 07 00:21:19.802489 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.802378 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws" event={"ID":"d9966cbe-974f-4429-b252-99a75813dfb6","Type":"ContainerDied","Data":"000e546a09e1e04a8d062c600bfaa77833951384a45f3d46bd1f96b76cc235c2"} Jul 07 00:21:19.802489 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.802398 2558 scope.go:117] "RemoveContainer" containerID="42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743" Jul 07 00:21:19.811591 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.811569 2558 scope.go:117] "RemoveContainer" containerID="42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743" Jul 07 00:21:19.811903 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:21:19.811873 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743\": container with ID starting with 42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743 not found: ID does not exist" containerID="42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743" Jul 07 00:21:19.811990 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.811915 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743"} err="failed to get container status \"42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743\": rpc error: code = NotFound desc = could not find container \"42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743\": container with ID starting with 42d33a9003be10c919d7be5919be98f92f63544f8b9402d8b570abc510af4743 not found: ID does not exist" Jul 07 00:21:19.822789 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.822760 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws"] Jul 07 00:21:19.829325 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.829302 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-91d197-57d774b5f8-xd9ws"] Jul 07 00:21:19.857821 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.857784 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9966cbe-974f-4429-b252-99a75813dfb6-maas-api-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:19.857821 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.857814 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9966cbe-974f-4429-b252-99a75813dfb6-openshift-service-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:19.857821 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:19.857825 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgfhd\" (UniqueName: \"kubernetes.io/projected/d9966cbe-974f-4429-b252-99a75813dfb6-kube-api-access-kgfhd\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:20.052737 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:20.052650 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9966cbe-974f-4429-b252-99a75813dfb6" path="/var/lib/kubelet/pods/d9966cbe-974f-4429-b252-99a75813dfb6/volumes" Jul 07 00:21:25.241223 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:25.241188 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:25.392362 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:25.392320 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:25.526781 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:25.526688 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x"] Jul 07 00:21:25.526977 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:25.526938 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" containerID="cri-o://6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d" gracePeriod=30 Jul 07 00:21:25.537673 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:25.537640 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:25.795560 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:25.795466 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:25.978472 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:25.978431 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:26.372208 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:26.372171 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:26.601730 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:26.601694 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:26.699277 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:26.699247 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:26.865234 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:26.865172 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:30.791101 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.791036 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:21:30.854399 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.854365 2558 generic.go:356] "Generic (PLEG): container finished" podID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerID="6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d" exitCode=0 Jul 07 00:21:30.854568 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.854420 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" Jul 07 00:21:30.854568 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.854427 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" event={"ID":"56364ead-957f-4aa3-b0b4-516a18f917b3","Type":"ContainerDied","Data":"6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d"} Jul 07 00:21:30.854568 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.854500 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x" event={"ID":"56364ead-957f-4aa3-b0b4-516a18f917b3","Type":"ContainerDied","Data":"cfcd7e9c8de1a850ed6e5ec33cdf7b381882d1d3718fc275ce3cf8c6abf0f52e"} Jul 07 00:21:30.854568 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.854522 2558 scope.go:117] "RemoveContainer" containerID="6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d" Jul 07 00:21:30.868689 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.868644 2558 scope.go:117] "RemoveContainer" containerID="284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20" Jul 07 00:21:30.877370 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.877346 2558 scope.go:117] "RemoveContainer" containerID="6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d" Jul 07 00:21:30.877691 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:21:30.877667 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d\": container with ID starting with 6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d not found: ID does not exist" containerID="6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d" Jul 07 00:21:30.877741 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.877702 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d"} err="failed to get container status \"6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d\": rpc error: code = NotFound desc = could not find container \"6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d\": container with ID starting with 6b8b149b802c0558bb295d669789ab631174a04a04b6336d9015baa7a8410b0d not found: ID does not exist" Jul 07 00:21:30.877741 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.877721 2558 scope.go:117] "RemoveContainer" containerID="284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20" Jul 07 00:21:30.878048 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:21:30.878030 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20\": container with ID starting with 284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20 not found: ID does not exist" containerID="284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20" Jul 07 00:21:30.878137 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.878050 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20"} err="failed to get container status \"284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20\": rpc error: code = NotFound desc = could not find container \"284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20\": container with ID starting with 284645b1abf7c87c59f8bba6f9b0303238c426173d2ea1f7293ba0218c6a9a20 not found: ID does not exist" Jul 07 00:21:30.958146 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958111 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-data\") pod \"56364ead-957f-4aa3-b0b4-516a18f917b3\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " Jul 07 00:21:30.958146 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958152 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-podinfo\") pod \"56364ead-957f-4aa3-b0b4-516a18f917b3\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " Jul 07 00:21:30.958351 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958179 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-socket\") pod \"56364ead-957f-4aa3-b0b4-516a18f917b3\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " Jul 07 00:21:30.958351 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958204 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-envoy\") pod \"56364ead-957f-4aa3-b0b4-516a18f917b3\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " Jul 07 00:21:30.958351 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958240 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qgb6\" (UniqueName: \"kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-kube-api-access-8qgb6\") pod \"56364ead-957f-4aa3-b0b4-516a18f917b3\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " Jul 07 00:21:30.958351 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958298 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56364ead-957f-4aa3-b0b4-516a18f917b3-istiod-ca-cert\") pod \"56364ead-957f-4aa3-b0b4-516a18f917b3\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " Jul 07 00:21:30.958351 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958337 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-token\") pod \"56364ead-957f-4aa3-b0b4-516a18f917b3\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " Jul 07 00:21:30.958612 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958391 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-certs\") pod \"56364ead-957f-4aa3-b0b4-516a18f917b3\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " Jul 07 00:21:30.958612 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958414 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-credential-socket\") pod \"56364ead-957f-4aa3-b0b4-516a18f917b3\" (UID: \"56364ead-957f-4aa3-b0b4-516a18f917b3\") " Jul 07 00:21:30.958612 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958542 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56364ead-957f-4aa3-b0b4-516a18f917b3" (UID: "56364ead-957f-4aa3-b0b4-516a18f917b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:21:30.958784 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958763 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:30.958784 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958767 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56364ead-957f-4aa3-b0b4-516a18f917b3" (UID: "56364ead-957f-4aa3-b0b4-516a18f917b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:21:30.958921 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.958824 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56364ead-957f-4aa3-b0b4-516a18f917b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56364ead-957f-4aa3-b0b4-516a18f917b3" (UID: "56364ead-957f-4aa3-b0b4-516a18f917b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:21:30.959187 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.959135 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56364ead-957f-4aa3-b0b4-516a18f917b3" (UID: "56364ead-957f-4aa3-b0b4-516a18f917b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:21:30.959528 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.959502 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "56364ead-957f-4aa3-b0b4-516a18f917b3" (UID: "56364ead-957f-4aa3-b0b4-516a18f917b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:21:30.960569 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.960544 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56364ead-957f-4aa3-b0b4-516a18f917b3" (UID: "56364ead-957f-4aa3-b0b4-516a18f917b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:21:30.960805 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.960788 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "56364ead-957f-4aa3-b0b4-516a18f917b3" (UID: "56364ead-957f-4aa3-b0b4-516a18f917b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:21:30.961137 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.961115 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56364ead-957f-4aa3-b0b4-516a18f917b3" (UID: "56364ead-957f-4aa3-b0b4-516a18f917b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:21:30.961137 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:30.961121 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-kube-api-access-8qgb6" (OuterVolumeSpecName: "kube-api-access-8qgb6") pod "56364ead-957f-4aa3-b0b4-516a18f917b3" (UID: "56364ead-957f-4aa3-b0b4-516a18f917b3"). InnerVolumeSpecName "kube-api-access-8qgb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:21:31.059656 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:31.059563 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-podinfo\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:31.059656 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:31.059596 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-envoy\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:31.059656 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:31.059608 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qgb6\" (UniqueName: \"kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-kube-api-access-8qgb6\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:31.059656 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:31.059619 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56364ead-957f-4aa3-b0b4-516a18f917b3-istiod-ca-cert\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:31.059656 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:31.059627 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-token\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:31.059656 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:31.059635 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-workload-certs\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:31.059656 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:31.059643 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-credential-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:31.059656 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:31.059650 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56364ead-957f-4aa3-b0b4-516a18f917b3-istio-data\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:21:31.177380 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:31.177335 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x"] Jul 07 00:21:31.182091 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:31.182059 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-91d197-openshift-default-84c8698d77-4zk8x"] Jul 07 00:21:32.051869 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:32.051810 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" path="/var/lib/kubelet/pods/56364ead-957f-4aa3-b0b4-516a18f917b3/volumes" Jul 07 00:21:32.112084 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:32.112047 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:32.222227 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:32.222192 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:32.441107 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:32.441071 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:32.633721 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:32.633687 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:32.983253 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:32.983219 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:33.300546 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.300462 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx"] Jul 07 00:21:33.300910 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.300776 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" Jul 07 00:21:33.300910 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.300787 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" Jul 07 00:21:33.300910 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.300799 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" Jul 07 00:21:33.300910 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.300806 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" Jul 07 00:21:33.300910 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.300820 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9966cbe-974f-4429-b252-99a75813dfb6" containerName="maas-api" Jul 07 00:21:33.300910 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.300825 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9966cbe-974f-4429-b252-99a75813dfb6" containerName="maas-api" Jul 07 00:21:33.301109 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.300929 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" Jul 07 00:21:33.301109 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.300938 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9966cbe-974f-4429-b252-99a75813dfb6" containerName="maas-api" Jul 07 00:21:33.301109 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.300944 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="56364ead-957f-4aa3-b0b4-516a18f917b3" containerName="istio-proxy" Jul 07 00:21:33.304969 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.304950 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.308420 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.308398 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a7bd37-openshift-default-dockercfg-s7b5x\"" Jul 07 00:21:33.323534 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.323499 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx"] Jul 07 00:21:33.345700 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.345659 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:33.445390 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.445356 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:33.479696 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.479649 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-token\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.479889 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.479706 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-envoy\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.479889 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.479772 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-data\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.479889 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.479809 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss2rd\" (UniqueName: \"kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-kube-api-access-ss2rd\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.479889 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.479833 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-socket\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.479889 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.479878 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97492cae-0cad-41a9-9905-7847c3bfec7b-istiod-ca-cert\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.480105 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.479928 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-credential-socket\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.480105 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.479972 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-certs\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.480105 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.480011 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-podinfo\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.581512 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.581409 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-credential-socket\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.581512 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.581459 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-certs\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.581512 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.581496 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-podinfo\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.581808 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.581537 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-token\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.581808 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.581584 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-envoy\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.581808 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.581620 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-data\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.581808 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.581645 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss2rd\" (UniqueName: \"kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-kube-api-access-ss2rd\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.581808 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.581673 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-socket\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.581808 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.581697 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97492cae-0cad-41a9-9905-7847c3bfec7b-istiod-ca-cert\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.582293 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.581948 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-certs\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.582293 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.582083 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-data\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.582293 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.582133 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-socket\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.582293 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.582172 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-credential-socket\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.582498 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.582445 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97492cae-0cad-41a9-9905-7847c3bfec7b-istiod-ca-cert\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.584122 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.584096 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-envoy\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.584248 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.584232 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-podinfo\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.591659 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.591633 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-token\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.591993 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.591972 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss2rd\" (UniqueName: \"kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-kube-api-access-ss2rd\") pod \"e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.615733 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.615691 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:33.766662 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.766547 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx"] Jul 07 00:21:33.769571 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:21:33.769539 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97492cae_0cad_41a9_9905_7847c3bfec7b.slice/crio-d827d917b2adbae3faaccc7ac6d34639fa1fb1897f5b8ba9f9934728887ef86c WatchSource:0}: Error finding container d827d917b2adbae3faaccc7ac6d34639fa1fb1897f5b8ba9f9934728887ef86c: Status 404 returned error can't find the container with id d827d917b2adbae3faaccc7ac6d34639fa1fb1897f5b8ba9f9934728887ef86c Jul 07 00:21:33.771724 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.771690 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:21:33.771792 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.771762 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:21:33.771834 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.771791 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:21:33.872013 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.871896 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" event={"ID":"97492cae-0cad-41a9-9905-7847c3bfec7b","Type":"ContainerStarted","Data":"4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed"} Jul 07 00:21:33.872217 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.872172 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" event={"ID":"97492cae-0cad-41a9-9905-7847c3bfec7b","Type":"ContainerStarted","Data":"d827d917b2adbae3faaccc7ac6d34639fa1fb1897f5b8ba9f9934728887ef86c"} Jul 07 00:21:33.893934 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:33.893880 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" podStartSLOduration=0.893841533 podStartE2EDuration="893.841533ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:33.892050954 +0000 UTC m=+1974.402211911" watchObservedRunningTime="2026-07-07 00:21:33.893841533 +0000 UTC m=+1974.404002489" Jul 07 00:21:34.616364 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:34.616322 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:34.621384 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:34.621357 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:34.877578 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:34.877492 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:34.878552 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:34.878533 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:35.351292 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:35.351253 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:35.448248 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:35.448210 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:35.796864 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:35.796807 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:35.866949 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:35.866915 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5"] Jul 07 00:21:35.870921 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:35.870895 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:35.873956 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:35.873933 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-a7bd37-serving-cert\"" Jul 07 00:21:35.885050 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:35.885013 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5"] Jul 07 00:21:35.886925 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:35.886899 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:35.954636 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:35.954596 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:36.004664 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.004626 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4186b64d-606e-4e89-86c1-14346d653000-maas-api-tls\") pod \"maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.004878 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.004678 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4186b64d-606e-4e89-86c1-14346d653000-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.004878 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.004815 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vqvg\" (UniqueName: \"kubernetes.io/projected/4186b64d-606e-4e89-86c1-14346d653000-kube-api-access-6vqvg\") pod \"maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.105821 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.105725 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vqvg\" (UniqueName: \"kubernetes.io/projected/4186b64d-606e-4e89-86c1-14346d653000-kube-api-access-6vqvg\") pod \"maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.105821 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.105810 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4186b64d-606e-4e89-86c1-14346d653000-maas-api-tls\") pod \"maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.106053 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.105842 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4186b64d-606e-4e89-86c1-14346d653000-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.106053 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:21:36.105948 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-a7bd37-serving-cert: secret "maas-api-e2e-shared-a-a7bd37-serving-cert" not found Jul 07 00:21:36.106053 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:21:36.106009 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4186b64d-606e-4e89-86c1-14346d653000-maas-api-tls podName:4186b64d-606e-4e89-86c1-14346d653000 nodeName:}" failed. No retries permitted until 2026-07-07 00:21:36.605991744 +0000 UTC m=+1977.116152679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4186b64d-606e-4e89-86c1-14346d653000-maas-api-tls") pod "maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" (UID: "4186b64d-606e-4e89-86c1-14346d653000") : secret "maas-api-e2e-shared-a-a7bd37-serving-cert" not found Jul 07 00:21:36.106519 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.106502 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4186b64d-606e-4e89-86c1-14346d653000-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.129260 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.129223 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vqvg\" (UniqueName: \"kubernetes.io/projected/4186b64d-606e-4e89-86c1-14346d653000-kube-api-access-6vqvg\") pod \"maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.217063 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.217003 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:36.380959 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.380799 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:36.481804 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.481758 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:36.612105 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.612062 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4186b64d-606e-4e89-86c1-14346d653000-maas-api-tls\") pod \"maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.614721 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.614690 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4186b64d-606e-4e89-86c1-14346d653000-maas-api-tls\") pod \"maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.783051 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.783015 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:36.949726 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:36.949700 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5"] Jul 07 00:21:36.952065 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:21:36.952034 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4186b64d_606e_4e89_86c1_14346d653000.slice/crio-a94e7f935b0ed3541e3bb88a40d84a40486ece8b897f1b2344ba130a7a7bf208 WatchSource:0}: Error finding container a94e7f935b0ed3541e3bb88a40d84a40486ece8b897f1b2344ba130a7a7bf208: Status 404 returned error can't find the container with id a94e7f935b0ed3541e3bb88a40d84a40486ece8b897f1b2344ba130a7a7bf208 Jul 07 00:21:37.895321 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:37.895286 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" event={"ID":"4186b64d-606e-4e89-86c1-14346d653000","Type":"ContainerStarted","Data":"32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca"} Jul 07 00:21:37.895542 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:37.895341 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" event={"ID":"4186b64d-606e-4e89-86c1-14346d653000","Type":"ContainerStarted","Data":"a94e7f935b0ed3541e3bb88a40d84a40486ece8b897f1b2344ba130a7a7bf208"} Jul 07 00:21:37.895542 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:37.895363 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:37.915239 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:37.915185 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" podStartSLOduration=2.544772836 podStartE2EDuration="2.915166707s" totalImagesPullingTime="370.393871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:37.913429786 +0000 UTC m=+1978.423590749" watchObservedRunningTime="2026-07-07 00:21:37.915166707 +0000 UTC m=+1978.425327695" Jul 07 00:21:40.387038 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:40.386999 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:40.640919 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:40.640798 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:40.981220 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:40.981189 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:41.281953 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:41.281869 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:43.562585 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.562553 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:43.632520 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.632487 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx"] Jul 07 00:21:43.635983 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.635958 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:43.639386 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.639353 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-26a9dd-serving-cert\"" Jul 07 00:21:43.649237 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.649207 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx"] Jul 07 00:21:43.666504 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.666472 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:43.675556 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.675517 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh87z\" (UniqueName: \"kubernetes.io/projected/03e37782-538f-414c-9aef-c22933c90743-kube-api-access-vh87z\") pod \"maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:43.675740 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.675591 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03e37782-538f-414c-9aef-c22933c90743-maas-api-tls\") pod \"maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:43.675740 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.675644 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03e37782-538f-414c-9aef-c22933c90743-openshift-service-ca\") pod \"maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:43.764051 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.764004 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:43.776213 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.776179 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03e37782-538f-414c-9aef-c22933c90743-openshift-service-ca\") pod \"maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:43.776373 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.776247 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh87z\" (UniqueName: \"kubernetes.io/projected/03e37782-538f-414c-9aef-c22933c90743-kube-api-access-vh87z\") pod \"maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:43.776373 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.776283 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03e37782-538f-414c-9aef-c22933c90743-maas-api-tls\") pod \"maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:43.779884 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.776998 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03e37782-538f-414c-9aef-c22933c90743-openshift-service-ca\") pod \"maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:43.782885 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.781377 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03e37782-538f-414c-9aef-c22933c90743-maas-api-tls\") pod \"maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:43.795885 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.795460 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh87z\" (UniqueName: \"kubernetes.io/projected/03e37782-538f-414c-9aef-c22933c90743-kube-api-access-vh87z\") pod \"maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:43.871373 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.871272 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:43.906000 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.905971 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:21:43.946754 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:43.946721 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:44.160189 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:44.160155 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx"] Jul 07 00:21:44.163600 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:21:44.163568 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e37782_538f_414c_9aef_c22933c90743.slice/crio-bc3b43ed13acaf8a201470ad440a6b84ff84492f1df4c06c5fdd336a46c340e1 WatchSource:0}: Error finding container bc3b43ed13acaf8a201470ad440a6b84ff84492f1df4c06c5fdd336a46c340e1: Status 404 returned error can't find the container with id bc3b43ed13acaf8a201470ad440a6b84ff84492f1df4c06c5fdd336a46c340e1 Jul 07 00:21:44.345301 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:44.345262 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:44.445397 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:44.445357 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:44.930930 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:44.930896 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" event={"ID":"03e37782-538f-414c-9aef-c22933c90743","Type":"ContainerStarted","Data":"72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f"} Jul 07 00:21:44.931333 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:44.930978 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" event={"ID":"03e37782-538f-414c-9aef-c22933c90743","Type":"ContainerStarted","Data":"bc3b43ed13acaf8a201470ad440a6b84ff84492f1df4c06c5fdd336a46c340e1"} Jul 07 00:21:44.931333 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:44.931022 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:44.951998 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:44.951932 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" podStartSLOduration=1.651703124 podStartE2EDuration="1.951912539s" totalImagesPullingTime="300.209415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:44.950242539 +0000 UTC m=+1985.460403546" watchObservedRunningTime="2026-07-07 00:21:44.951912539 +0000 UTC m=+1985.462073495" Jul 07 00:21:48.052078 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:48.052043 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:49.878674 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:49.878627 2558 patch_prober.go:28] interesting pod/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 00:21:49.879239 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:49.878693 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 00:21:49.957898 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:49.957838 2558 generic.go:356] "Generic (PLEG): container finished" podID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerID="4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed" exitCode=0 Jul 07 00:21:49.958080 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:49.957916 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" event={"ID":"97492cae-0cad-41a9-9905-7847c3bfec7b","Type":"ContainerDied","Data":"4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed"} Jul 07 00:21:49.958315 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:49.958300 2558 scope.go:117] "RemoveContainer" containerID="4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed" Jul 07 00:21:49.959328 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:49.959296 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:21:49.959428 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:49.959383 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:21:49.959489 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:49.959457 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 00:21:50.942104 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:50.942069 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:21:50.964999 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:50.964959 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" event={"ID":"97492cae-0cad-41a9-9905-7847c3bfec7b","Type":"ContainerStarted","Data":"12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27"} Jul 07 00:21:51.616666 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:51.616617 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:51.616923 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:51.616888 2558 patch_prober.go:28] interesting pod/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 00:21:51.617013 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:51.616963 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 00:21:52.616595 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:52.616552 2558 patch_prober.go:28] interesting pod/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 00:21:52.617021 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:52.616653 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 00:21:53.616745 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:53.616704 2558 patch_prober.go:28] interesting pod/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 00:21:53.617175 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:53.616776 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 00:21:54.616745 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:54.616707 2558 patch_prober.go:28] interesting pod/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 00:21:54.617168 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:54.616775 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 00:21:55.620593 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:55.620560 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:55.621014 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:55.620832 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:55.621554 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:55.621537 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:21:58.120112 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:58.120076 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:58.217481 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:21:58.217449 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:06.185522 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:22:06.185488 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:06.290094 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:22:06.290045 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:10.629478 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:22:10.629444 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:30.332167 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:22:30.332130 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:23:40.100383 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:23:40.100354 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:23:40.105282 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:23:40.105254 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:24:19.521562 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:24:19.521517 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:24:26.189436 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:24:26.189399 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:24:27.112539 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:24:27.112496 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:24:36.493220 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:24:36.493136 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:25:18.190027 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:25:18.189989 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:25:20.028154 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:25:20.028109 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:25:26.882070 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:25:26.882035 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:25:26.944890 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:25:26.944829 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:25:58.075780 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:25:58.075733 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:25:59.091337 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:25:59.091294 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:26:14.755906 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:26:14.755800 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:26:30.370283 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:26:30.370242 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:26:36.065338 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:26:36.065290 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:26:36.162978 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:26:36.162940 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:26:40.928398 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:26:40.928357 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:26:58.936372 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:26:58.936337 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:26:59.036180 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:26:59.036142 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:06.889652 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:06.889610 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:06.989958 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:06.989920 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:19.952375 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:19.952324 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:25.187079 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:25.187036 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:26.099725 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:26.099686 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:27.159136 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:27.159095 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:27.587862 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:27.587823 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:27.691205 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:27.691168 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:27.794029 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:27.793987 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:27.878973 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:27.878885 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5"] Jul 07 00:27:27.879186 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:27.879137 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" podUID="4186b64d-606e-4e89-86c1-14346d653000" containerName="maas-api" containerID="cri-o://32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca" gracePeriod=30 Jul 07 00:27:27.917962 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:27.917930 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:27:28.137467 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.137399 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:27:28.226600 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.226562 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4186b64d-606e-4e89-86c1-14346d653000-openshift-service-ca\") pod \"4186b64d-606e-4e89-86c1-14346d653000\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " Jul 07 00:27:28.227098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.226659 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vqvg\" (UniqueName: \"kubernetes.io/projected/4186b64d-606e-4e89-86c1-14346d653000-kube-api-access-6vqvg\") pod \"4186b64d-606e-4e89-86c1-14346d653000\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " Jul 07 00:27:28.227098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.226712 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4186b64d-606e-4e89-86c1-14346d653000-maas-api-tls\") pod \"4186b64d-606e-4e89-86c1-14346d653000\" (UID: \"4186b64d-606e-4e89-86c1-14346d653000\") " Jul 07 00:27:28.227098 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.226997 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4186b64d-606e-4e89-86c1-14346d653000-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4186b64d-606e-4e89-86c1-14346d653000" (UID: "4186b64d-606e-4e89-86c1-14346d653000"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:27:28.228998 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.228967 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4186b64d-606e-4e89-86c1-14346d653000-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4186b64d-606e-4e89-86c1-14346d653000" (UID: "4186b64d-606e-4e89-86c1-14346d653000"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:27:28.229088 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.229036 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4186b64d-606e-4e89-86c1-14346d653000-kube-api-access-6vqvg" (OuterVolumeSpecName: "kube-api-access-6vqvg") pod "4186b64d-606e-4e89-86c1-14346d653000" (UID: "4186b64d-606e-4e89-86c1-14346d653000"). InnerVolumeSpecName "kube-api-access-6vqvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:27:28.327672 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.327629 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vqvg\" (UniqueName: \"kubernetes.io/projected/4186b64d-606e-4e89-86c1-14346d653000-kube-api-access-6vqvg\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:27:28.327672 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.327666 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4186b64d-606e-4e89-86c1-14346d653000-maas-api-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:27:28.327672 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.327681 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4186b64d-606e-4e89-86c1-14346d653000-openshift-service-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:27:28.507265 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.507226 2558 generic.go:356] "Generic (PLEG): container finished" podID="4186b64d-606e-4e89-86c1-14346d653000" containerID="32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca" exitCode=0 Jul 07 00:27:28.507451 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.507285 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" Jul 07 00:27:28.507451 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.507309 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" event={"ID":"4186b64d-606e-4e89-86c1-14346d653000","Type":"ContainerDied","Data":"32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca"} Jul 07 00:27:28.507451 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.507361 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5" event={"ID":"4186b64d-606e-4e89-86c1-14346d653000","Type":"ContainerDied","Data":"a94e7f935b0ed3541e3bb88a40d84a40486ece8b897f1b2344ba130a7a7bf208"} Jul 07 00:27:28.507451 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.507383 2558 scope.go:117] "RemoveContainer" containerID="32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca" Jul 07 00:27:28.517302 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.517282 2558 scope.go:117] "RemoveContainer" containerID="32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca" Jul 07 00:27:28.517582 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:27:28.517559 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca\": container with ID starting with 32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca not found: ID does not exist" containerID="32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca" Jul 07 00:27:28.517654 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.517592 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca"} err="failed to get container status \"32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca\": rpc error: code = NotFound desc = could not find container \"32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca\": container with ID starting with 32c8ebadbfacd340d538268fb626f4027aad987d2be5629a9a0130745591a9ca not found: ID does not exist" Jul 07 00:27:28.530446 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.530418 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5"] Jul 07 00:27:28.533951 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:28.533929 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a7bd37-67974cf6-fwrl5"] Jul 07 00:27:30.051410 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:30.051344 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4186b64d-606e-4e89-86c1-14346d653000" path="/var/lib/kubelet/pods/4186b64d-606e-4e89-86c1-14346d653000/volumes" Jul 07 00:27:32.836582 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:27:32.836501 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:09.615155 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:09.615112 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:09.877704 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:09.877619 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx"] Jul 07 00:28:09.877905 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:09.877823 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" containerID="cri-o://12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27" gracePeriod=30 Jul 07 00:28:09.902424 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:09.902393 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:10.114372 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:10.114317 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:10.271729 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:10.271685 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:10.623952 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:10.623871 2558 patch_prober.go:28] interesting pod/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 00:28:10.623952 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:10.623939 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 00:28:10.688386 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:10.688348 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:10.787490 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:10.787455 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:10.889291 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:10.889201 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:10.944754 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:10.944720 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx"] Jul 07 00:28:10.944974 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:10.944949 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" podUID="03e37782-538f-414c-9aef-c22933c90743" containerName="maas-api" containerID="cri-o://72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f" gracePeriod=30 Jul 07 00:28:10.987940 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:10.987896 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:11.200199 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.200168 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:28:11.317695 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.317665 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh87z\" (UniqueName: \"kubernetes.io/projected/03e37782-538f-414c-9aef-c22933c90743-kube-api-access-vh87z\") pod \"03e37782-538f-414c-9aef-c22933c90743\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " Jul 07 00:28:11.317695 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.317708 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03e37782-538f-414c-9aef-c22933c90743-openshift-service-ca\") pod \"03e37782-538f-414c-9aef-c22933c90743\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " Jul 07 00:28:11.318009 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.317743 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03e37782-538f-414c-9aef-c22933c90743-maas-api-tls\") pod \"03e37782-538f-414c-9aef-c22933c90743\" (UID: \"03e37782-538f-414c-9aef-c22933c90743\") " Jul 07 00:28:11.318137 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.318112 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03e37782-538f-414c-9aef-c22933c90743-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "03e37782-538f-414c-9aef-c22933c90743" (UID: "03e37782-538f-414c-9aef-c22933c90743"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:28:11.320076 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.320046 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03e37782-538f-414c-9aef-c22933c90743-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "03e37782-538f-414c-9aef-c22933c90743" (UID: "03e37782-538f-414c-9aef-c22933c90743"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:28:11.320189 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.320078 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03e37782-538f-414c-9aef-c22933c90743-kube-api-access-vh87z" (OuterVolumeSpecName: "kube-api-access-vh87z") pod "03e37782-538f-414c-9aef-c22933c90743" (UID: "03e37782-538f-414c-9aef-c22933c90743"). InnerVolumeSpecName "kube-api-access-vh87z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:28:11.419355 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.419301 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh87z\" (UniqueName: \"kubernetes.io/projected/03e37782-538f-414c-9aef-c22933c90743-kube-api-access-vh87z\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:11.419355 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.419347 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03e37782-538f-414c-9aef-c22933c90743-openshift-service-ca\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:11.419355 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.419358 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03e37782-538f-414c-9aef-c22933c90743-maas-api-tls\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:11.704066 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.704035 2558 generic.go:356] "Generic (PLEG): container finished" podID="03e37782-538f-414c-9aef-c22933c90743" containerID="72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f" exitCode=0 Jul 07 00:28:11.704564 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.704084 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" Jul 07 00:28:11.704564 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.704113 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" event={"ID":"03e37782-538f-414c-9aef-c22933c90743","Type":"ContainerDied","Data":"72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f"} Jul 07 00:28:11.704564 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.704173 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx" event={"ID":"03e37782-538f-414c-9aef-c22933c90743","Type":"ContainerDied","Data":"bc3b43ed13acaf8a201470ad440a6b84ff84492f1df4c06c5fdd336a46c340e1"} Jul 07 00:28:11.704564 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.704196 2558 scope.go:117] "RemoveContainer" containerID="72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f" Jul 07 00:28:11.713793 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.713776 2558 scope.go:117] "RemoveContainer" containerID="72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f" Jul 07 00:28:11.714144 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:28:11.714115 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f\": container with ID starting with 72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f not found: ID does not exist" containerID="72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f" Jul 07 00:28:11.714226 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.714160 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f"} err="failed to get container status \"72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f\": rpc error: code = NotFound desc = could not find container \"72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f\": container with ID starting with 72d12cff055ed9a47a8b24bd240f4cb437a76b657153233d2cf49ccb01031a5f not found: ID does not exist" Jul 07 00:28:11.722345 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:28:11.722317 2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e37782_538f_414c_9aef_c22933c90743.slice/crio-bc3b43ed13acaf8a201470ad440a6b84ff84492f1df4c06c5fdd336a46c340e1\": RecentStats: unable to find data in memory cache]" Jul 07 00:28:11.732326 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.727991 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx"] Jul 07 00:28:11.735880 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:11.735837 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-26a9dd-59d67d4df9-vxftx"] Jul 07 00:28:12.051579 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:12.051498 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03e37782-538f-414c-9aef-c22933c90743" path="/var/lib/kubelet/pods/03e37782-538f-414c-9aef-c22933c90743/volumes" Jul 07 00:28:15.135516 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.135491 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:28:15.254766 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.254732 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-data\") pod \"97492cae-0cad-41a9-9905-7847c3bfec7b\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " Jul 07 00:28:15.254972 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.254776 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-podinfo\") pod \"97492cae-0cad-41a9-9905-7847c3bfec7b\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " Jul 07 00:28:15.254972 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.254798 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-credential-socket\") pod \"97492cae-0cad-41a9-9905-7847c3bfec7b\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " Jul 07 00:28:15.254972 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.254826 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97492cae-0cad-41a9-9905-7847c3bfec7b-istiod-ca-cert\") pod \"97492cae-0cad-41a9-9905-7847c3bfec7b\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " Jul 07 00:28:15.254972 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.254876 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-envoy\") pod \"97492cae-0cad-41a9-9905-7847c3bfec7b\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " Jul 07 00:28:15.254972 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.254901 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-certs\") pod \"97492cae-0cad-41a9-9905-7847c3bfec7b\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " Jul 07 00:28:15.254972 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.254952 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss2rd\" (UniqueName: \"kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-kube-api-access-ss2rd\") pod \"97492cae-0cad-41a9-9905-7847c3bfec7b\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " Jul 07 00:28:15.255272 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.254997 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-token\") pod \"97492cae-0cad-41a9-9905-7847c3bfec7b\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " Jul 07 00:28:15.255272 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.255018 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-socket\") pod \"97492cae-0cad-41a9-9905-7847c3bfec7b\" (UID: \"97492cae-0cad-41a9-9905-7847c3bfec7b\") " Jul 07 00:28:15.255272 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.255172 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "97492cae-0cad-41a9-9905-7847c3bfec7b" (UID: "97492cae-0cad-41a9-9905-7847c3bfec7b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:28:15.255427 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.255318 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-credential-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:15.255427 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.255392 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "97492cae-0cad-41a9-9905-7847c3bfec7b" (UID: "97492cae-0cad-41a9-9905-7847c3bfec7b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:28:15.255427 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.255402 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "97492cae-0cad-41a9-9905-7847c3bfec7b" (UID: "97492cae-0cad-41a9-9905-7847c3bfec7b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:28:15.255427 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.255398 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97492cae-0cad-41a9-9905-7847c3bfec7b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "97492cae-0cad-41a9-9905-7847c3bfec7b" (UID: "97492cae-0cad-41a9-9905-7847c3bfec7b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:28:15.255907 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.255869 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-data" (OuterVolumeSpecName: "istio-data") pod "97492cae-0cad-41a9-9905-7847c3bfec7b" (UID: "97492cae-0cad-41a9-9905-7847c3bfec7b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:28:15.257375 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.257354 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "97492cae-0cad-41a9-9905-7847c3bfec7b" (UID: "97492cae-0cad-41a9-9905-7847c3bfec7b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:28:15.257462 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.257427 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-token" (OuterVolumeSpecName: "istio-token") pod "97492cae-0cad-41a9-9905-7847c3bfec7b" (UID: "97492cae-0cad-41a9-9905-7847c3bfec7b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:28:15.257868 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.257817 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "97492cae-0cad-41a9-9905-7847c3bfec7b" (UID: "97492cae-0cad-41a9-9905-7847c3bfec7b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:28:15.257975 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.257883 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-kube-api-access-ss2rd" (OuterVolumeSpecName: "kube-api-access-ss2rd") pod "97492cae-0cad-41a9-9905-7847c3bfec7b" (UID: "97492cae-0cad-41a9-9905-7847c3bfec7b"). InnerVolumeSpecName "kube-api-access-ss2rd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:28:15.355963 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.355920 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss2rd\" (UniqueName: \"kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-kube-api-access-ss2rd\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:15.355963 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.355952 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-token\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:15.355963 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.355967 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-socket\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:15.356218 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.355978 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-data\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:15.356218 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.355990 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-podinfo\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:15.356218 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.356000 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97492cae-0cad-41a9-9905-7847c3bfec7b-istiod-ca-cert\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:15.356218 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.356011 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-istio-envoy\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:15.356218 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.356023 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97492cae-0cad-41a9-9905-7847c3bfec7b-workload-certs\") on node \"ip-10-0-130-213.ec2.internal\" DevicePath \"\"" Jul 07 00:28:15.723673 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.723632 2558 generic.go:356] "Generic (PLEG): container finished" podID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerID="12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27" exitCode=0 Jul 07 00:28:15.723872 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.723683 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" Jul 07 00:28:15.723872 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.723715 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" event={"ID":"97492cae-0cad-41a9-9905-7847c3bfec7b","Type":"ContainerDied","Data":"12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27"} Jul 07 00:28:15.723872 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.723762 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx" event={"ID":"97492cae-0cad-41a9-9905-7847c3bfec7b","Type":"ContainerDied","Data":"d827d917b2adbae3faaccc7ac6d34639fa1fb1897f5b8ba9f9934728887ef86c"} Jul 07 00:28:15.723872 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.723784 2558 scope.go:117] "RemoveContainer" containerID="12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27" Jul 07 00:28:15.733469 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.733448 2558 scope.go:117] "RemoveContainer" containerID="4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed" Jul 07 00:28:15.741427 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.741405 2558 scope.go:117] "RemoveContainer" containerID="12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27" Jul 07 00:28:15.741681 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:28:15.741657 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27\": container with ID starting with 12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27 not found: ID does not exist" containerID="12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27" Jul 07 00:28:15.741792 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.741688 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27"} err="failed to get container status \"12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27\": rpc error: code = NotFound desc = could not find container \"12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27\": container with ID starting with 12235b147821a995a4395bf58574ec7927a2d39bd4f195097224e0d6c99c5f27 not found: ID does not exist" Jul 07 00:28:15.741792 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.741705 2558 scope.go:117] "RemoveContainer" containerID="4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed" Jul 07 00:28:15.742018 ip-10-0-130-213 kubenswrapper[2558]: E0707 00:28:15.741998 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed\": container with ID starting with 4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed not found: ID does not exist" containerID="4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed" Jul 07 00:28:15.742067 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.742026 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed"} err="failed to get container status \"4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed\": rpc error: code = NotFound desc = could not find container \"4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed\": container with ID starting with 4e85ebc7c3fef9bc8f092fe4b87714db6e8094387ae2eda08e4b72c8f8fdc1ed not found: ID does not exist" Jul 07 00:28:15.745682 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.745662 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx"] Jul 07 00:28:15.748983 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:15.748963 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a7bd37-openshift-default-6fcc6f4547-gngpx"] Jul 07 00:28:16.052352 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:16.052265 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" path="/var/lib/kubelet/pods/97492cae-0cad-41a9-9905-7847c3bfec7b/volumes" Jul 07 00:28:16.562441 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:16.562409 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:16.663116 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:16.663079 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:16.837887 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:16.837765 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:17.073605 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:17.073574 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:22.308937 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:22.308894 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6b58c8fc8d-bfggs_7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c/maas-api/0.log" Jul 07 00:28:22.648778 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:22.648692 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6d86fcfb77-62mlh_143497c2-c2b8-4a7c-8bd4-8b6645368ff6/manager/0.log" Jul 07 00:28:22.990748 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:22.990721 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64648fd55c-rcl68_71c36dbf-02fd-4924-a7ff-fd7d4490f2e2/copy-manifests/0.log" Jul 07 00:28:22.997699 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:22.997675 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64648fd55c-rcl68_71c36dbf-02fd-4924-a7ff-fd7d4490f2e2/manager/0.log" Jul 07 00:28:23.368504 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:23.368405 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-rpsqs_dacb6431-f5d1-4e25-a6ac-2ae78d6813eb/postgres/0.log" Jul 07 00:28:25.472537 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:25.472504 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-phpx2_ad21f8cc-7347-469b-9006-488d24ea3427/manager/0.log" Jul 07 00:28:25.941053 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:25.941003 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gdl6n_f4542275-7cb0-4a4b-8f52-6f9acc49c7bb/discovery/0.log" Jul 07 00:28:26.164248 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:26.164220 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c55d5ff9-scg7x_9f554f49-1343-445a-a60a-49781ba73cac/kube-auth-proxy/0.log" Jul 07 00:28:26.274795 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:26.274709 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zh8gc_86ddbcaa-9010-4191-a96d-1218f5c44b33/istio-proxy/1.log" Jul 07 00:28:26.391051 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:26.391011 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5trzj_d44a3bbc-40b6-417a-b1fe-1bd13330ed47/payload-pre-processing/0.log" Jul 07 00:28:26.509837 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:26.509805 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wwn72_d27bb777-193f-4ee0-9117-3c0d57eea737/payload-processing/0.log" Jul 07 00:28:26.959488 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:26.959455 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8l96f_23d4b580-7008-4b78-ab1f-63f4ac558f83/main/0.log" Jul 07 00:28:27.188570 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:27.188543 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-nnrtq_2587b7a8-36b1-4f10-b724-23dcc98fbafc/main/0.log" Jul 07 00:28:27.522389 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:27.522360 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-psgpl_4978afe4-69a4-42c3-afba-3d7b99d6182c/main/0.log" Jul 07 00:28:31.738428 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:31.738383 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:31.803237 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:31.803206 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" limit=1024 Jul 07 00:28:34.374963 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:34.374932 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8dl25_3dee8e4b-8ebb-470a-aaac-a610ba92c6dd/global-pull-secret-syncer/0.log" Jul 07 00:28:34.497728 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:34.497698 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2nd7r_fb6985f2-1b61-47fd-a1b0-17429a573c7d/konnectivity-agent/0.log" Jul 07 00:28:34.596052 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:34.596025 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-213.ec2.internal_373575230b88bb4ec2f11847d42c51bd/haproxy/0.log" Jul 07 00:28:40.128417 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:40.128372 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:28:40.134578 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:40.134554 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z7v2_707c4468-3606-41ee-8a87-dc5764875fa5/ovn-acl-logging/0.log" Jul 07 00:28:40.387165 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:40.387082 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-phpx2_ad21f8cc-7347-469b-9006-488d24ea3427/manager/0.log" Jul 07 00:28:42.548066 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:42.548031 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bnlwl_87b29f4b-efd0-490b-9611-1f1d01633846/node-exporter/0.log" Jul 07 00:28:42.572972 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:42.572942 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bnlwl_87b29f4b-efd0-490b-9611-1f1d01633846/kube-rbac-proxy/0.log" Jul 07 00:28:42.592962 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:42.592938 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bnlwl_87b29f4b-efd0-490b-9611-1f1d01633846/init-textfile/0.log" Jul 07 00:28:44.782006 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.781967 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j"] Jul 07 00:28:44.782485 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.782325 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" Jul 07 00:28:44.782485 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.782338 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" Jul 07 00:28:44.782485 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.782357 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03e37782-538f-414c-9aef-c22933c90743" containerName="maas-api" Jul 07 00:28:44.782485 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.782362 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="03e37782-538f-414c-9aef-c22933c90743" containerName="maas-api" Jul 07 00:28:44.782485 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.782372 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4186b64d-606e-4e89-86c1-14346d653000" containerName="maas-api" Jul 07 00:28:44.782485 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.782378 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4186b64d-606e-4e89-86c1-14346d653000" containerName="maas-api" Jul 07 00:28:44.782485 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.782427 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="03e37782-538f-414c-9aef-c22933c90743" containerName="maas-api" Jul 07 00:28:44.782485 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.782438 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" Jul 07 00:28:44.782485 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.782443 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="97492cae-0cad-41a9-9905-7847c3bfec7b" containerName="istio-proxy" Jul 07 00:28:44.782485 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.782449 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4186b64d-606e-4e89-86c1-14346d653000" containerName="maas-api" Jul 07 00:28:44.785359 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.785342 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.788068 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.788046 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f7d67\"/\"kube-root-ca.crt\"" Jul 07 00:28:44.788200 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.788046 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f7d67\"/\"openshift-service-ca.crt\"" Jul 07 00:28:44.788200 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.788049 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f7d67\"/\"default-dockercfg-bpjqr\"" Jul 07 00:28:44.794646 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.794618 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j"] Jul 07 00:28:44.796297 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.796261 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-lib-modules\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.796397 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.796322 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck5gj\" (UniqueName: \"kubernetes.io/projected/d15d89a4-ea54-411a-8435-79908ab80c44-kube-api-access-ck5gj\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.796397 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.796351 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-proc\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.796397 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.796373 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-podres\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.796524 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.796492 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-sys\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.897166 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.897103 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-sys\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.897368 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.897190 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-lib-modules\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.897368 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.897244 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck5gj\" (UniqueName: \"kubernetes.io/projected/d15d89a4-ea54-411a-8435-79908ab80c44-kube-api-access-ck5gj\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.897368 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.897254 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-sys\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.897368 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.897268 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-proc\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.897368 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.897286 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-podres\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.897368 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.897305 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-lib-modules\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.897368 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.897365 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-proc\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.897639 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.897432 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d15d89a4-ea54-411a-8435-79908ab80c44-podres\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:44.906009 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:44.905974 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck5gj\" (UniqueName: \"kubernetes.io/projected/d15d89a4-ea54-411a-8435-79908ab80c44-kube-api-access-ck5gj\") pod \"perf-node-gather-daemonset-gxs5j\" (UID: \"d15d89a4-ea54-411a-8435-79908ab80c44\") " pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:45.096698 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:45.096612 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:45.236214 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:45.236188 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j"] Jul 07 00:28:45.239347 ip-10-0-130-213 kubenswrapper[2558]: W0707 00:28:45.239314 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd15d89a4_ea54_411a_8435_79908ab80c44.slice/crio-485ea7df867367bb282d27fe2a9c450643e248330324458783146d3d5a7e9da1 WatchSource:0}: Error finding container 485ea7df867367bb282d27fe2a9c450643e248330324458783146d3d5a7e9da1: Status 404 returned error can't find the container with id 485ea7df867367bb282d27fe2a9c450643e248330324458783146d3d5a7e9da1 Jul 07 00:28:45.862499 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:45.862467 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" event={"ID":"d15d89a4-ea54-411a-8435-79908ab80c44","Type":"ContainerStarted","Data":"fc796fd9d3a1a5a1e0c10ef383c8197ffce2da8049ee3950dcefbd10a1eb8dcb"} Jul 07 00:28:45.862977 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:45.862518 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" event={"ID":"d15d89a4-ea54-411a-8435-79908ab80c44","Type":"ContainerStarted","Data":"485ea7df867367bb282d27fe2a9c450643e248330324458783146d3d5a7e9da1"} Jul 07 00:28:45.862977 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:45.862568 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:45.882172 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:45.882114 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" podStartSLOduration=1.882094961 podStartE2EDuration="1.882094961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:28:45.880513023 +0000 UTC m=+2406.390673982" watchObservedRunningTime="2026-07-07 00:28:45.882094961 +0000 UTC m=+2406.392255916" Jul 07 00:28:47.585181 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:47.585144 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s8nvv_9dfd8b83-8e48-4bcd-8449-bed29a1a166f/dns/0.log" Jul 07 00:28:47.628066 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:47.628034 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s8nvv_9dfd8b83-8e48-4bcd-8449-bed29a1a166f/kube-rbac-proxy/0.log" Jul 07 00:28:47.827239 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:47.827204 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wxmxp_4423aa72-95c7-4610-9d94-76b3f1c2b76e/dns-node-resolver/0.log" Jul 07 00:28:48.475018 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:48.474986 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-brsvk_9ff9b33f-ec62-4d90-80dc-745c2ffa9498/node-ca/0.log" Jul 07 00:28:49.597097 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:49.597062 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gdl6n_f4542275-7cb0-4a4b-8f52-6f9acc49c7bb/discovery/0.log" Jul 07 00:28:49.640709 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:49.640680 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c55d5ff9-scg7x_9f554f49-1343-445a-a60a-49781ba73cac/kube-auth-proxy/0.log" Jul 07 00:28:49.669234 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:49.669204 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zh8gc_86ddbcaa-9010-4191-a96d-1218f5c44b33/istio-proxy/0.log" Jul 07 00:28:49.698093 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:49.698058 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zh8gc_86ddbcaa-9010-4191-a96d-1218f5c44b33/istio-proxy/1.log" Jul 07 00:28:49.728522 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:49.728495 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5trzj_d44a3bbc-40b6-417a-b1fe-1bd13330ed47/payload-pre-processing/0.log" Jul 07 00:28:49.777868 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:49.777818 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wwn72_d27bb777-193f-4ee0-9117-3c0d57eea737/payload-processing/0.log" Jul 07 00:28:50.561174 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:50.561142 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q2rzk_cdf73de7-5618-48db-9e38-37e2b82f372b/serve-healthcheck-canary/0.log" Jul 07 00:28:51.154841 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:51.154807 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c44w6_42e0dccb-3536-4e4c-8aa7-fe4c34741997/kube-rbac-proxy/0.log" Jul 07 00:28:51.175473 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:51.175445 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c44w6_42e0dccb-3536-4e4c-8aa7-fe4c34741997/exporter/0.log" Jul 07 00:28:51.195755 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:51.195724 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c44w6_42e0dccb-3536-4e4c-8aa7-fe4c34741997/extractor/0.log" Jul 07 00:28:51.877402 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:51.877372 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f7d67/perf-node-gather-daemonset-gxs5j" Jul 07 00:28:53.796211 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:53.796178 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6b58c8fc8d-bfggs_7c2f2b8b-e50f-4dda-b8e0-f9ef82262e1c/maas-api/0.log" Jul 07 00:28:53.942010 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:53.941969 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6d86fcfb77-62mlh_143497c2-c2b8-4a7c-8bd4-8b6645368ff6/manager/0.log" Jul 07 00:28:54.099673 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:54.099585 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64648fd55c-rcl68_71c36dbf-02fd-4924-a7ff-fd7d4490f2e2/manager/0.log" Jul 07 00:28:54.125668 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:54.125646 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64648fd55c-rcl68_71c36dbf-02fd-4924-a7ff-fd7d4490f2e2/copy-manifests/0.log" Jul 07 00:28:54.345658 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:54.345629 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-rpsqs_dacb6431-f5d1-4e25-a6ac-2ae78d6813eb/postgres/0.log" Jul 07 00:28:55.659133 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:55.659105 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gdl6n_f4542275-7cb0-4a4b-8f52-6f9acc49c7bb/discovery/0.log" Jul 07 00:28:55.688066 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:55.688038 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c55d5ff9-scg7x_9f554f49-1343-445a-a60a-49781ba73cac/kube-auth-proxy/0.log" Jul 07 00:28:55.706988 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:55.706945 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zh8gc_86ddbcaa-9010-4191-a96d-1218f5c44b33/istio-proxy/0.log" Jul 07 00:28:55.733651 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:55.733618 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zh8gc_86ddbcaa-9010-4191-a96d-1218f5c44b33/istio-proxy/1.log" Jul 07 00:28:55.755587 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:55.755557 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5trzj_d44a3bbc-40b6-417a-b1fe-1bd13330ed47/payload-pre-processing/0.log" Jul 07 00:28:55.757793 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:55.757769 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-cqmpm_dd49fb7b-5ec3-442e-800f-0d94e7ee53d4/openshift-lws-operator/0.log" Jul 07 00:28:55.792551 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:55.792521 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wwn72_d27bb777-193f-4ee0-9117-3c0d57eea737/payload-processing/0.log" Jul 07 00:28:57.124129 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:57.124091 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gdl6n_f4542275-7cb0-4a4b-8f52-6f9acc49c7bb/discovery/0.log" Jul 07 00:28:57.153197 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:57.153167 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c55d5ff9-scg7x_9f554f49-1343-445a-a60a-49781ba73cac/kube-auth-proxy/0.log" Jul 07 00:28:57.188468 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:57.188438 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zh8gc_86ddbcaa-9010-4191-a96d-1218f5c44b33/istio-proxy/0.log" Jul 07 00:28:57.217028 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:57.216991 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zh8gc_86ddbcaa-9010-4191-a96d-1218f5c44b33/istio-proxy/1.log" Jul 07 00:28:57.238001 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:57.237966 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5trzj_d44a3bbc-40b6-417a-b1fe-1bd13330ed47/payload-pre-processing/0.log" Jul 07 00:28:57.277274 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:28:57.277229 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wwn72_d27bb777-193f-4ee0-9117-3c0d57eea737/payload-processing/0.log" Jul 07 00:29:01.974451 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:29:01.974417 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-24nhh_847d0bbc-bde0-4006-ad49-b87007f8923c/kube-multus/0.log" Jul 07 00:29:02.278421 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:29:02.278342 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw7t9_4ebb3c3f-eda3-4c28-987b-11627a176748/kube-multus-additional-cni-plugins/0.log" Jul 07 00:29:02.299423 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:29:02.299395 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw7t9_4ebb3c3f-eda3-4c28-987b-11627a176748/egress-router-binary-copy/0.log" Jul 07 00:29:02.319565 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:29:02.319541 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw7t9_4ebb3c3f-eda3-4c28-987b-11627a176748/cni-plugins/0.log" Jul 07 00:29:02.340667 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:29:02.340635 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw7t9_4ebb3c3f-eda3-4c28-987b-11627a176748/bond-cni-plugin/0.log" Jul 07 00:29:02.382149 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:29:02.382122 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw7t9_4ebb3c3f-eda3-4c28-987b-11627a176748/routeoverride-cni/0.log" Jul 07 00:29:02.438625 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:29:02.438598 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw7t9_4ebb3c3f-eda3-4c28-987b-11627a176748/whereabouts-cni-bincopy/0.log" Jul 07 00:29:02.490303 ip-10-0-130-213 kubenswrapper[2558]: I0707 00:29:02.490271 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw7t9_4ebb3c3f-eda3-4c28-987b-11627a176748/whereabouts-cni/0.log"