Jul 13 07:00:19.649756 ip-10-0-133-57 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 13 07:00:19.649765 ip-10-0-133-57 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 13 07:00:19.649772 ip-10-0-133-57 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 13 07:00:19.650093 ip-10-0-133-57 systemd[1]: Failed to start Kubernetes Kubelet. Jul 13 07:00:29.746268 ip-10-0-133-57 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 13 07:00:29.746289 ip-10-0-133-57 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 4506b633243a4e5ca41e341e71974438 -- Jul 13 07:02:26.703452 ip-10-0-133-57 systemd[1]: Starting Kubernetes Kubelet... Jul 13 07:02:27.213736 ip-10-0-133-57 systemd[1]: Started Kubernetes Kubelet. Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.133964 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140287 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140298 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140301 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140305 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140308 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140311 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140314 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:27.222866 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140317 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140320 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140322 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140325 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140328 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140330 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140333 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140338 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140342 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140346 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140349 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140352 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140355 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140358 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140361 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140363 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140366 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140369 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140372 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:27.223755 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140375 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140381 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140385 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140387 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140390 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140395 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140398 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140401 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140404 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140407 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140410 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140413 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140416 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140419 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140422 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140424 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140428 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140430 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140433 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140436 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:27.224378 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140439 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140442 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140444 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140447 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140449 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140452 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140455 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140457 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140460 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140463 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140466 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140468 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140471 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140473 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140476 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140478 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140481 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140483 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140486 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140488 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:27.225045 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140491 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140494 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140497 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140499 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140502 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140505 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140507 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140510 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140512 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140515 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140517 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140520 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140523 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140525 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140529 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140534 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140537 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140540 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140543 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:27.225753 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140545 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140547 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140550 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140552 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140555 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140557 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140560 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140562 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140565 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140567 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140570 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140572 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140575 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.140578 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143018 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143026 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143030 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143033 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143037 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143040 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:27.226509 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143043 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143045 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143048 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143051 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143053 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143056 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143058 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143061 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143064 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143067 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143069 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143072 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143074 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143077 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143079 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143082 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143084 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143087 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143089 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143092 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:27.227358 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143094 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143097 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143099 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143102 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143104 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143107 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143110 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143113 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143116 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143119 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143124 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143128 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143131 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143134 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143138 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143142 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143145 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143148 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143151 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:27.228414 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143153 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143157 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143159 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143162 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143165 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143167 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143170 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143172 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143175 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143178 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143180 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143185 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143189 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143192 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143194 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143197 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143216 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143219 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143221 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:27.229463 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143226 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143229 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143232 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143234 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143237 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143240 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143243 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143245 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143248 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143250 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143253 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143256 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143258 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143261 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143264 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143266 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143269 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143272 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143274 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143276 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:27.230516 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143279 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143281 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143284 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143286 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143289 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143291 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143293 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143296 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143299 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143301 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143303 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143306 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143310 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143313 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143315 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143696 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143701 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143705 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143708 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143711 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:27.231539 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143714 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143716 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143719 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143722 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143725 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143727 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143730 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143732 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143735 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143738 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143740 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143744 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143747 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143749 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143753 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143757 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143759 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143762 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143765 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:27.232368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143768 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143771 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143773 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143776 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143778 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143782 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143785 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143788 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143791 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143794 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143797 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143800 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143803 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143806 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143809 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143812 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143814 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143817 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143819 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143822 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:27.232988 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143824 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143828 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143832 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143834 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143836 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143839 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143842 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143844 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143846 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143849 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143851 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143854 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143856 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143859 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143861 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143865 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143872 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143877 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143880 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:27.233803 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143882 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143885 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143888 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143890 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143893 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143895 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143898 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143900 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143903 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143905 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143908 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143910 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143913 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143915 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143918 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143921 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143923 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143926 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143928 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143931 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:27.234636 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143934 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143936 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143939 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143941 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143944 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143946 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143949 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143951 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143953 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143957 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143961 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143964 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143966 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143969 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143971 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.143974 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144037 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144043 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144049 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144053 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144058 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 07:02:27.235454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144061 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144065 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144069 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144072 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144076 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144079 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144082 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144086 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144089 2583 flags.go:64] FLAG: --cgroup-root="" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144092 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144095 2583 flags.go:64] FLAG: --client-ca-file="" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144098 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144101 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144105 2583 flags.go:64] FLAG: --cluster-domain="" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144108 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144111 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144114 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144117 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144120 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144123 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144128 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144133 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144135 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144138 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 07:02:27.236533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144141 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144144 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144147 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144151 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144154 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144157 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144159 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144162 2583 flags.go:64] FLAG: --enable-server="true" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144165 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144169 2583 flags.go:64] FLAG: --event-burst="100" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144172 2583 flags.go:64] FLAG: --event-qps="50" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144175 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144179 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144182 2583 flags.go:64] FLAG: --eviction-hard="" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144185 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144188 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144191 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144194 2583 flags.go:64] FLAG: --eviction-soft="" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144197 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144214 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144217 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144220 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144223 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144225 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144228 2583 flags.go:64] FLAG: --feature-gates="" Jul 13 07:02:27.237686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144232 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144235 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144238 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144242 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144247 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144250 2583 flags.go:64] FLAG: --help="false" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144253 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144256 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144258 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144261 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144265 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144268 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144271 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144274 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144277 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144280 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144282 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144285 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144288 2583 flags.go:64] FLAG: --kube-reserved="" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144291 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144294 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144297 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144299 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144302 2583 flags.go:64] FLAG: --lock-file="" Jul 13 07:02:27.238838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144305 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144308 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144311 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144316 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144319 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144322 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144325 2583 flags.go:64] FLAG: --logging-format="text" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144328 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144331 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144334 2583 flags.go:64] FLAG: --manifest-url="" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144337 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144342 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144347 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144351 2583 flags.go:64] FLAG: --max-pods="110" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144354 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144357 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144359 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144362 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144365 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144368 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144371 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144378 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144381 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144384 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 07:02:27.240030 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144387 2583 flags.go:64] FLAG: --pod-cidr="" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144390 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144395 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144398 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144401 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144404 2583 flags.go:64] FLAG: --port="10250" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144406 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144409 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-035c6155940ef6fe7" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144412 2583 flags.go:64] FLAG: --qos-reserved="" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144416 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144419 2583 flags.go:64] FLAG: --register-node="true" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144421 2583 flags.go:64] FLAG: --register-with-taints="" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144425 2583 flags.go:64] FLAG: --registry-burst="10" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144427 2583 flags.go:64] FLAG: --registry-qps="5" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144430 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144433 2583 flags.go:64] FLAG: --reserved-memory="" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144437 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144439 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144442 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144445 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144450 2583 flags.go:64] FLAG: --runonce="false" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144453 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144456 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144459 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144462 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 07:02:27.429002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144465 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144467 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144470 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144473 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144476 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144479 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144482 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144485 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144488 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144490 2583 flags.go:64] FLAG: --system-cgroups="" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144493 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144498 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144501 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144504 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144508 2583 flags.go:64] FLAG: --tls-min-version="" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144511 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144513 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144520 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144522 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144525 2583 flags.go:64] FLAG: --v="2" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144529 2583 flags.go:64] FLAG: --version="false" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144533 2583 flags.go:64] FLAG: --vmodule="" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144536 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144539 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144644 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:27.447995 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144648 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144651 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144656 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144659 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144662 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144666 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144668 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144671 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144674 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144676 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144679 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144682 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144684 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144687 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144689 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144692 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144694 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144697 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144700 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144702 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:27.448938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144705 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144707 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144710 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144712 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144715 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144718 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144720 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144723 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144725 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144728 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144731 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144733 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144736 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144740 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144745 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144748 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144751 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144754 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144757 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144759 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:27.449621 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144762 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144765 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144767 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144770 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144774 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144777 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144781 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144785 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144788 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144791 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144793 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144796 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144798 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144801 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144803 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144806 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144808 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144810 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144813 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:27.450353 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144816 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144819 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144821 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144824 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144826 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144828 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144831 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144836 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144839 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144841 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144844 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144846 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144849 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144851 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144854 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144856 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144859 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144862 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144864 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144867 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:27.451022 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144869 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144872 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144875 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144877 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144880 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144882 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144885 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144887 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144890 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144893 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144895 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144897 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144901 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144903 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144906 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144908 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144910 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144913 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:27.452100 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.144915 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.144925 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.153618 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.153636 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.154521 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.154528 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154589 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154594 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154598 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154601 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154604 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154607 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154609 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154612 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154615 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:27.452712 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154617 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154620 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154624 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154629 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154632 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154636 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154640 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154643 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154646 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154648 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154651 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154654 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154656 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154660 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154664 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154667 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154670 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154673 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154676 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:27.453261 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154679 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154688 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154691 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154694 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154697 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154700 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154703 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154705 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154708 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154711 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154714 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154716 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154719 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154722 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154725 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154728 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154731 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154733 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154736 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154738 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:27.453837 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154741 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154744 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154746 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154751 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154754 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154757 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154759 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154762 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154765 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154768 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154771 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154774 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154777 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154779 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154787 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154790 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154793 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154795 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154798 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:27.454494 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154800 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154803 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154805 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154808 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154810 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154813 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154816 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154826 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154829 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154831 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154834 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154837 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154839 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154842 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154845 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154847 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154851 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154854 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154856 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154859 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:27.676685 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154861 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154864 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154866 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154870 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154873 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154875 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154878 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154880 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154889 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154892 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154894 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.154897 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.154902 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155030 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155035 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:27.677619 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155038 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155041 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155043 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155046 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155049 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155051 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155053 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155056 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155058 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155061 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155064 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155066 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155068 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155071 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155075 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155077 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155080 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155084 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155088 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:27.678175 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155091 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155096 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155099 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155102 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155105 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155107 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155110 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155118 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155121 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155124 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155126 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155129 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155131 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155134 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155136 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155138 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155141 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155143 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155146 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:27.678872 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155148 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155152 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155156 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155159 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155162 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155166 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155169 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155171 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155176 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155178 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155181 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155183 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155186 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155189 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155192 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155195 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155197 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155213 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155216 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:27.679500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155218 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155221 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155229 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155232 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155235 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155237 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155240 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155243 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155245 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155248 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155250 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155252 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155255 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155257 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155260 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155262 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155265 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155267 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155270 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155272 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:27.680092 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155274 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155278 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155281 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155283 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155286 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155288 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155291 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155295 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155297 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155300 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155302 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155305 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155307 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155310 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155312 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155322 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155325 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155328 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155330 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:27.680910 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:27.155333 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.155344 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.155451 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.157440 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.158923 2583 server.go:1019] "Starting client certificate rotation" Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.159021 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.159060 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.161783 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.166384 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.177986 2583 log.go:25] "Validated CRI v1 runtime API" Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.180033 2583 log.go:25] "Validated CRI v1 image API" Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.180817 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 07:02:27.681491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.183313 2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a357a905-b3ef-41a6-9993-85a269ab6afd:/dev/nvme0n1p3 e4660bdd-8813-4cf6-8e30-c0b177dd39bc:/dev/nvme0n1p4] Jul 13 07:02:27.681968 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.183337 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 07:02:27.681968 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.187498 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.189574 2583 manager.go:216] Machine: {Timestamp:2026-07-13 07:02:27.188076914 +0000 UTC m=+0.362580715 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099218 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec280b9b7547ddae1f7326a0b3915e00 SystemUUID:ec280b9b-7547-ddae-1f73-26a0b3915e00 BootID:4506b633-243a-4e5c-a41e-341e71974438 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:02:37:2e:e7:45 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:02:37:2e:e7:45 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:60:8e:a7:9c:67 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.189700 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.189802 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.192840 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.192867 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-57.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.193515 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.193524 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.193857 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.194606 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.195794 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.200076 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.201976 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.202589 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.202743 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.202879 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.203026 2583 kubelet.go:398] "Adding apiserver pod source" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.203244 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 07:02:27.682257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.203667 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.203686 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.206151 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208004 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208035 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208198 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208239 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208252 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208266 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208283 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208296 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208911 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208919 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208926 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208931 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208939 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.208947 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.209809 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.209816 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.212955 2583 server.go:1268] "Started kubelet" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.213427 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.213677 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.216661 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.217312 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.217576 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-57.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.217806 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.218096 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bv6xk" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.220300 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.220622 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.222907 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.223849 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-57.ec2.internal.18c1c76116286a75 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-57.ec2.internal,UID:ip-10-0-133-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-57.ec2.internal,},FirstTimestamp:2026-07-13 07:02:27.212929653 +0000 UTC m=+0.387433451,LastTimestamp:2026-07-13 07:02:27.212929653 +0000 UTC m=+0.387433451,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-57.ec2.internal,}" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.225270 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bv6xk" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.226060 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.226121 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.226131 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.226637 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.226638 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.226683 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.226753 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.226763 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 13 07:02:27.682887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.226898 2583 factory.go:55] Registering systemd factory Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.226910 2583 factory.go:223] Registration of the systemd container factory successfully Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.227156 2583 factory.go:153] Registering CRI-O factory Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.227176 2583 factory.go:223] Registration of the crio container factory successfully Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.227232 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.227252 2583 factory.go:103] Registering Raw factory Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.227289 2583 manager.go:1196] Started watching for new ooms in manager Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.227615 2583 manager.go:349] Starting recovery of all containers Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.240381 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.243069 2583 manager.go:354] Recovery completed Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.243524 2583 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-57.ec2.internal\" not found" node="ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.247083 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.248843 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.248871 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.248884 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.249291 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.249299 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.249318 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.249427 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.250222 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.250249 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.250262 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.250924 2583 policy_none.go:49] "None policy: Start" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.250938 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.250950 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.252061 2583 policy_none.go:47] "Start" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.293760 2583 manager.go:337] "Starting Device Plugin manager" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.293784 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.293792 2583 server.go:87] "Starting device plugin registration server" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.293959 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.294089 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.294172 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.294317 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.294329 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.294705 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.294754 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.394389 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.395694 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.395718 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.395727 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.395748 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.401835 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.401851 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-57.ec2.internal\": node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.417926 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.420631 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.421798 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.421829 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.421844 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.421872 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 07:02:27.684020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.424216 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.522997 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal"] Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.523012 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.523069 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.524763 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.524788 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.524799 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.525939 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.526049 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.526073 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.526747 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.526762 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.526772 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.526783 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.526787 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.526795 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.528042 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.528063 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.528179 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cc3a3928a1786918c202ceb8745343f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal\" (UID: \"cc3a3928a1786918c202ceb8745343f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.528204 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc3a3928a1786918c202ceb8745343f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal\" (UID: \"cc3a3928a1786918c202ceb8745343f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.528238 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b573b3797697c17ef8fefe9494186a99-config\") pod \"kube-apiserver-proxy-ip-10-0-133-57.ec2.internal\" (UID: \"b573b3797697c17ef8fefe9494186a99\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.537228 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.537255 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.537266 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.551815 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-57.ec2.internal\" not found" node="ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.557563 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-57.ec2.internal\" not found" node="ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.623922 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.628517 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cc3a3928a1786918c202ceb8745343f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal\" (UID: \"cc3a3928a1786918c202ceb8745343f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.628541 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc3a3928a1786918c202ceb8745343f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal\" (UID: \"cc3a3928a1786918c202ceb8745343f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.685430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.628560 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b573b3797697c17ef8fefe9494186a99-config\") pod \"kube-apiserver-proxy-ip-10-0-133-57.ec2.internal\" (UID: \"b573b3797697c17ef8fefe9494186a99\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.686236 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.628612 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b573b3797697c17ef8fefe9494186a99-config\") pod \"kube-apiserver-proxy-ip-10-0-133-57.ec2.internal\" (UID: \"b573b3797697c17ef8fefe9494186a99\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.686236 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.628624 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cc3a3928a1786918c202ceb8745343f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal\" (UID: \"cc3a3928a1786918c202ceb8745343f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.686236 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.628645 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc3a3928a1786918c202ceb8745343f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal\" (UID: \"cc3a3928a1786918c202ceb8745343f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.877778 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.725429 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:27.877778 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.831726 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:27.877778 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.854571 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" Jul 13 07:02:27.877778 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:27.861394 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal" Jul 13 07:02:28.074133 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:27.932842 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:28.074133 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:28.033922 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:28.192125 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:28.134728 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:28.192125 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.157794 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 07:02:28.192125 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.157903 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 07:02:28.192125 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.157958 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 07:02:28.383552 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.228763 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 06:57:28 +0000 UTC" deadline="2028-01-17 01:58:26.918997441 +0000 UTC" Jul 13 07:02:28.383552 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.228788 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13266h55m58.690212515s" Jul 13 07:02:28.383552 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:28.234974 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:28.383552 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:28.335065 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-57.ec2.internal\" not found" Jul 13 07:02:28.383552 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.370242 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:28.598003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.427178 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal" Jul 13 07:02:28.598003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.438098 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 07:02:28.598003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.440035 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" Jul 13 07:02:28.598003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.441360 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:28.598003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.452476 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 07:02:28.598003 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:28.541145 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc3a3928a1786918c202ceb8745343f1.slice/crio-5d5b94431212ab51d8758b65463c616e8054347067e10b7abca8a216246d92d8 WatchSource:0}: Error finding container 5d5b94431212ab51d8758b65463c616e8054347067e10b7abca8a216246d92d8: Status 404 returned error can't find the container with id 5d5b94431212ab51d8758b65463c616e8054347067e10b7abca8a216246d92d8 Jul 13 07:02:28.598003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:28.547217 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:02:28.598003 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:28.548491 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb573b3797697c17ef8fefe9494186a99.slice/crio-9314a91ad6245ffa5db02ed7fd74a304473bdddb6daf1e7e390a547e39aed042 WatchSource:0}: Error finding container 9314a91ad6245ffa5db02ed7fd74a304473bdddb6daf1e7e390a547e39aed042: Status 404 returned error can't find the container with id 9314a91ad6245ffa5db02ed7fd74a304473bdddb6daf1e7e390a547e39aed042 Jul 13 07:02:29.226566 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.207439 2583 apiserver.go:52] "Watching apiserver" Jul 13 07:02:29.228858 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.227107 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.243009 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.243492 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q","openshift-cluster-node-tuning-operator/tuned-hdnpv","openshift-dns/node-resolver-ztprt","openshift-network-diagnostics/network-check-target-f8xn4","kube-system/global-pull-secret-syncer-b2qc9","openshift-image-registry/node-ca-sxk2l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal","openshift-multus/multus-additional-cni-plugins-l8fxn","openshift-multus/multus-jkjh6","openshift-multus/network-metrics-daemon-jzrkj","openshift-network-operator/iptables-alerter-4hqg6","openshift-ovn-kubernetes/ovnkube-node-lvl52","kube-system/konnectivity-agent-msp7f","kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal"] Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.245100 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.245239 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.246236 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.247149 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.247458 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.250266 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.250298 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.250378 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.250469 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.250495 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lqh7p\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.250476 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.250481 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.252387 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.253638 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.255041 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xvm9m\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.255108 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.255241 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.255389 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cj4fd\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.255473 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.255043 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.255759 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.256910 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.256964 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9mrd4\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.257114 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.257291 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.257401 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.257477 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.326057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.257496 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.258133 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.258298 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f4kdj\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.258317 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.258452 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.259779 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.261585 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.261770 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.261823 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4cjjx\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.261774 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.262118 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.262140 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.262291 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cgsfg\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.262576 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.263207 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.263243 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sclkx\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.263548 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.263592 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.264375 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.264650 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.264688 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.264923 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.268294 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c6nz8\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.268863 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.269671 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.269905 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kl6d4" Jul 13 07:02:29.327752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.287250 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kl6d4" Jul 13 07:02:29.330110 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.327948 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 07:02:29.367925 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.367896 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b6d741e6-fdd4-442e-9940-2588e3ef9419-dbus\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:29.368042 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.367940 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrbmz\" (UniqueName: \"kubernetes.io/projected/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-kube-api-access-hrbmz\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.368042 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.367965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-sys-fs\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.368042 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.367991 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-run-ovn\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.368042 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368018 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.368239 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368056 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-os-release\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.368239 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368081 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-conf-dir\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.368239 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368109 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb02a1e3-ebbd-4628-a045-f883d47208a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.368239 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368135 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-tuned\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.368239 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368159 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb02a1e3-ebbd-4628-a045-f883d47208a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.368239 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368193 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-run-ovn-kubernetes\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.368550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368236 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fd93e728-989f-4151-934f-ae52670e5531-ovnkube-script-lib\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.368550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368288 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4kzt\" (UniqueName: \"kubernetes.io/projected/a68e2b3a-fc60-4be7-9bf1-664317d72b94-kube-api-access-h4kzt\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:29.368550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368333 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-run-k8s-cni-cncf-io\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.368550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368377 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-lib-modules\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.368550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368417 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-run-openvswitch\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.368550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368445 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad5f621b-f27f-4abb-a5be-557e0bbceb86-host\") pod \"node-ca-sxk2l\" (UID: \"ad5f621b-f27f-4abb-a5be-557e0bbceb86\") " pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.368550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368472 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-system-cni-dir\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.368550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368499 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-etc-openvswitch\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.368550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368526 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fd93e728-989f-4151-934f-ae52670e5531-ovnkube-config\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368558 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-sysconfig\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368599 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-tmp\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368631 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-registration-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368661 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqkpr\" (UniqueName: \"kubernetes.io/projected/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-kube-api-access-pqkpr\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368686 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-systemd-units\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368710 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-var-lib-openvswitch\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368751 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-socket-dir-parent\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368793 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-os-release\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368828 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-sys\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368858 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0296e96-219b-41af-8adb-e7d65fbf5bdb-iptables-alerter-script\") pod \"iptables-alerter-4hqg6\" (UID: \"f0296e96-219b-41af-8adb-e7d65fbf5bdb\") " pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368901 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368937 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-var-lib-cni-bin\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368955 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-hostroot\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368971 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-etc-kubernetes\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.368998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.368986 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-cnibin\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369011 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb02a1e3-ebbd-4628-a045-f883d47208a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369027 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0296e96-219b-41af-8adb-e7d65fbf5bdb-host-slash\") pod \"iptables-alerter-4hqg6\" (UID: \"f0296e96-219b-41af-8adb-e7d65fbf5bdb\") " pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369046 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-slash\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369062 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-node-log\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369084 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-var-lib-kubelet\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369099 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-run-multus-certs\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369114 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-system-cni-dir\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369150 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd93e728-989f-4151-934f-ae52670e5531-ovn-node-metrics-cert\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369180 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369206 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-modprobe-d\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369249 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369272 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-log-socket\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369296 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-cni-bin\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369323 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-cni-netd\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369347 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1295416-be49-4b38-9602-0c28e9e90055-agent-certs\") pod \"konnectivity-agent-msp7f\" (UID: \"b1295416-be49-4b38-9602-0c28e9e90055\") " pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:29.369758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369371 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-var-lib-kubelet\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369394 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-device-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369420 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369443 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-socket-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369468 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-run-netns\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369492 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fd93e728-989f-4151-934f-ae52670e5531-env-overrides\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369526 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxgd4\" (UniqueName: \"kubernetes.io/projected/ad5f621b-f27f-4abb-a5be-557e0bbceb86-kube-api-access-lxgd4\") pod \"node-ca-sxk2l\" (UID: \"ad5f621b-f27f-4abb-a5be-557e0bbceb86\") " pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369551 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f663e38a-cc1c-4b21-a53a-c40b2ace16c5-tmp-dir\") pod \"node-resolver-ztprt\" (UID: \"f663e38a-cc1c-4b21-a53a-c40b2ace16c5\") " pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369575 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg4v2\" (UniqueName: \"kubernetes.io/projected/f663e38a-cc1c-4b21-a53a-c40b2ace16c5-kube-api-access-dg4v2\") pod \"node-resolver-ztprt\" (UID: \"f663e38a-cc1c-4b21-a53a-c40b2ace16c5\") " pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369606 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-sysctl-d\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369630 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-host\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369655 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7jzl\" (UniqueName: \"kubernetes.io/projected/fd93e728-989f-4151-934f-ae52670e5531-kube-api-access-c7jzl\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369711 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1295416-be49-4b38-9602-0c28e9e90055-konnectivity-ca\") pod \"konnectivity-agent-msp7f\" (UID: \"b1295416-be49-4b38-9602-0c28e9e90055\") " pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369761 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-cni-dir\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369812 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-cnibin\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369866 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.370555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369899 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6slnd\" (UniqueName: \"kubernetes.io/projected/fb02a1e3-ebbd-4628-a045-f883d47208a6-kube-api-access-6slnd\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369931 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6cv2\" (UniqueName: \"kubernetes.io/projected/f0296e96-219b-41af-8adb-e7d65fbf5bdb-kube-api-access-d6cv2\") pod \"iptables-alerter-4hqg6\" (UID: \"f0296e96-219b-41af-8adb-e7d65fbf5bdb\") " pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.369975 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-run\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370010 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-run-netns\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370037 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-var-lib-cni-multus\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370064 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-kubelet\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370103 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-kubernetes\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370144 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-systemd\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-run-systemd\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370201 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad5f621b-f27f-4abb-a5be-557e0bbceb86-serviceca\") pod \"node-ca-sxk2l\" (UID: \"ad5f621b-f27f-4abb-a5be-557e0bbceb86\") " pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370240 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c75397dd-0fb0-4607-9645-ce375ecc785d-cni-binary-copy\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370275 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-daemon-config\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370301 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-sysctl-conf\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.371347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.370327 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2lt7\" (UniqueName: \"kubernetes.io/projected/c75397dd-0fb0-4607-9645-ce375ecc785d-kube-api-access-h2lt7\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.377407 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.377383 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f663e38a-cc1c-4b21-a53a-c40b2ace16c5-hosts-file\") pod \"node-resolver-ztprt\" (UID: \"f663e38a-cc1c-4b21-a53a-c40b2ace16c5\") " pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.377513 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.377444 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llzn5\" (UniqueName: \"kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5\") pod \"network-check-target-f8xn4\" (UID: \"beb19682-944b-47da-b871-27e8f6dd449d\") " pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:29.377513 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.377471 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b6d741e6-fdd4-442e-9940-2588e3ef9419-kubelet-config\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:29.428639 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.428592 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal" event={"ID":"b573b3797697c17ef8fefe9494186a99","Type":"ContainerStarted","Data":"9314a91ad6245ffa5db02ed7fd74a304473bdddb6daf1e7e390a547e39aed042"} Jul 13 07:02:29.431457 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.431430 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" event={"ID":"cc3a3928a1786918c202ceb8745343f1","Type":"ContainerStarted","Data":"5d5b94431212ab51d8758b65463c616e8054347067e10b7abca8a216246d92d8"} Jul 13 07:02:29.477764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-etc-kubernetes\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.477764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477703 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-cnibin\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.477764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477730 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb02a1e3-ebbd-4628-a045-f883d47208a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.477764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477755 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0296e96-219b-41af-8adb-e7d65fbf5bdb-host-slash\") pod \"iptables-alerter-4hqg6\" (UID: \"f0296e96-219b-41af-8adb-e7d65fbf5bdb\") " pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477782 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-slash\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477818 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-node-log\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477820 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0296e96-219b-41af-8adb-e7d65fbf5bdb-host-slash\") pod \"iptables-alerter-4hqg6\" (UID: \"f0296e96-219b-41af-8adb-e7d65fbf5bdb\") " pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477842 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-var-lib-kubelet\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477866 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-run-multus-certs\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477879 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-etc-kubernetes\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477902 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-system-cni-dir\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477931 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd93e728-989f-4151-934f-ae52670e5531-ovn-node-metrics-cert\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477958 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477972 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-slash\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477984 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-modprobe-d\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478023 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-system-cni-dir\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.478043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478019 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478068 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-log-socket\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478092 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478096 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-cni-bin\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.477929 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-node-log\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478129 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-cni-netd\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478183 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-log-socket\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478324 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-modprobe-d\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478380 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb02a1e3-ebbd-4628-a045-f883d47208a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.478407 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478465 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-cnibin\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.478473 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret podName:b6d741e6-fdd4-442e-9940-2588e3ef9419 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:29.978451684 +0000 UTC m=+3.152955486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret") pod "global-pull-secret-syncer-b2qc9" (UID: "b6d741e6-fdd4-442e-9940-2588e3ef9419") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478508 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-var-lib-kubelet\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478513 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478548 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-cni-bin\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478598 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-cni-netd\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478652 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1295416-be49-4b38-9602-0c28e9e90055-agent-certs\") pod \"konnectivity-agent-msp7f\" (UID: \"b1295416-be49-4b38-9602-0c28e9e90055\") " pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:29.478813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-var-lib-kubelet\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478718 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-run-multus-certs\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478728 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-device-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478753 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478780 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-socket-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478807 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-run-netns\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478848 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fd93e728-989f-4151-934f-ae52670e5531-env-overrides\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478874 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxgd4\" (UniqueName: \"kubernetes.io/projected/ad5f621b-f27f-4abb-a5be-557e0bbceb86-kube-api-access-lxgd4\") pod \"node-ca-sxk2l\" (UID: \"ad5f621b-f27f-4abb-a5be-557e0bbceb86\") " pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478899 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f663e38a-cc1c-4b21-a53a-c40b2ace16c5-tmp-dir\") pod \"node-resolver-ztprt\" (UID: \"f663e38a-cc1c-4b21-a53a-c40b2ace16c5\") " pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478923 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg4v2\" (UniqueName: \"kubernetes.io/projected/f663e38a-cc1c-4b21-a53a-c40b2ace16c5-kube-api-access-dg4v2\") pod \"node-resolver-ztprt\" (UID: \"f663e38a-cc1c-4b21-a53a-c40b2ace16c5\") " pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478946 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-sysctl-d\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.478970 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-host\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479000 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7jzl\" (UniqueName: \"kubernetes.io/projected/fd93e728-989f-4151-934f-ae52670e5531-kube-api-access-c7jzl\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479025 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1295416-be49-4b38-9602-0c28e9e90055-konnectivity-ca\") pod \"konnectivity-agent-msp7f\" (UID: \"b1295416-be49-4b38-9602-0c28e9e90055\") " pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479049 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-cni-dir\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479068 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-var-lib-kubelet\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479128 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-cnibin\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.479661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479182 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-device-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479237 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479359 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-socket-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479073 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-cnibin\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479406 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479435 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6slnd\" (UniqueName: \"kubernetes.io/projected/fb02a1e3-ebbd-4628-a045-f883d47208a6-kube-api-access-6slnd\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479464 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6cv2\" (UniqueName: \"kubernetes.io/projected/f0296e96-219b-41af-8adb-e7d65fbf5bdb-kube-api-access-d6cv2\") pod \"iptables-alerter-4hqg6\" (UID: \"f0296e96-219b-41af-8adb-e7d65fbf5bdb\") " pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479490 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-run\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479518 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-run-netns\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479546 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-var-lib-cni-multus\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479543 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fd93e728-989f-4151-934f-ae52670e5531-env-overrides\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479605 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-kubelet\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479616 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-var-lib-cni-multus\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479637 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-kubernetes\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479679 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-systemd\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479705 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-run-systemd\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad5f621b-f27f-4abb-a5be-557e0bbceb86-serviceca\") pod \"node-ca-sxk2l\" (UID: \"ad5f621b-f27f-4abb-a5be-557e0bbceb86\") " pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.480491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.479758 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480207 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c75397dd-0fb0-4607-9645-ce375ecc785d-cni-binary-copy\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480236 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-daemon-config\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480268 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-sysctl-conf\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480310 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-run\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480318 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2lt7\" (UniqueName: \"kubernetes.io/projected/c75397dd-0fb0-4607-9645-ce375ecc785d-kube-api-access-h2lt7\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f663e38a-cc1c-4b21-a53a-c40b2ace16c5-hosts-file\") pod \"node-resolver-ztprt\" (UID: \"f663e38a-cc1c-4b21-a53a-c40b2ace16c5\") " pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480363 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-run-netns\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480370 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzn5\" (UniqueName: \"kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5\") pod \"network-check-target-f8xn4\" (UID: \"beb19682-944b-47da-b871-27e8f6dd449d\") " pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480398 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b6d741e6-fdd4-442e-9940-2588e3ef9419-kubelet-config\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480430 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b6d741e6-fdd4-442e-9940-2588e3ef9419-dbus\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480457 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrbmz\" (UniqueName: \"kubernetes.io/projected/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-kube-api-access-hrbmz\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480477 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-kubernetes\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480483 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-sys-fs\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480518 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-run-ovn\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480545 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-sys-fs\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480552 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.481371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480579 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-os-release\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480600 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-systemd\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480605 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-conf-dir\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480650 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-run-systemd\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480679 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb02a1e3-ebbd-4628-a045-f883d47208a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-tuned\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480734 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb02a1e3-ebbd-4628-a045-f883d47208a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480763 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-run-ovn-kubernetes\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480798 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fd93e728-989f-4151-934f-ae52670e5531-ovnkube-script-lib\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480860 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4kzt\" (UniqueName: \"kubernetes.io/projected/a68e2b3a-fc60-4be7-9bf1-664317d72b94-kube-api-access-h4kzt\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480892 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-run-k8s-cni-cncf-io\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.480917 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-lib-modules\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.481522 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-run-openvswitch\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.481583 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad5f621b-f27f-4abb-a5be-557e0bbceb86-host\") pod \"node-ca-sxk2l\" (UID: \"ad5f621b-f27f-4abb-a5be-557e0bbceb86\") " pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.481619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-system-cni-dir\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.482178 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.481651 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-etc-openvswitch\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.483580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483552 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fd93e728-989f-4151-934f-ae52670e5531-ovnkube-config\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.483685 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483624 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-sysconfig\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.483685 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483657 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-tmp\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.483770 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483695 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-registration-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.483770 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483734 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqkpr\" (UniqueName: \"kubernetes.io/projected/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-kube-api-access-pqkpr\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.483839 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483772 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-systemd-units\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.483839 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483806 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-var-lib-openvswitch\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.483916 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483835 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-socket-dir-parent\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.483916 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483872 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-os-release\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.483916 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483906 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-sys\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.484066 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483939 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0296e96-219b-41af-8adb-e7d65fbf5bdb-iptables-alerter-script\") pod \"iptables-alerter-4hqg6\" (UID: \"f0296e96-219b-41af-8adb-e7d65fbf5bdb\") " pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.484066 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.483979 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:29.484066 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.484012 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-var-lib-cni-bin\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.484066 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.484048 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-hostroot\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.484288 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.484142 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-hostroot\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.484288 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.484206 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-sysconfig\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.484909 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.484889 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd93e728-989f-4151-934f-ae52670e5531-ovn-node-metrics-cert\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485025 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-socket-dir-parent\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485140 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-systemd-units\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485148 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-sys\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485309 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-cni-dir\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485369 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-var-lib-openvswitch\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485482 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-sysctl-d\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485550 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-host\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485767 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0296e96-219b-41af-8adb-e7d65fbf5bdb-iptables-alerter-script\") pod \"iptables-alerter-4hqg6\" (UID: \"f0296e96-219b-41af-8adb-e7d65fbf5bdb\") " pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485844 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-run-netns\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485856 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-var-lib-cni-bin\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485908 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485945 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-kubelet\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.485977 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-run-ovn\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.486052 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1295416-be49-4b38-9602-0c28e9e90055-agent-certs\") pod \"konnectivity-agent-msp7f\" (UID: \"b1295416-be49-4b38-9602-0c28e9e90055\") " pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.486124 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-registration-dir\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.486197 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb02a1e3-ebbd-4628-a045-f883d47208a6-os-release\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.486337 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.486305 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:29.487215 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.486364 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs podName:a68e2b3a-fc60-4be7-9bf1-664317d72b94 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:29.986339607 +0000 UTC m=+3.160843403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs") pod "network-metrics-daemon-jzrkj" (UID: "a68e2b3a-fc60-4be7-9bf1-664317d72b94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:29.487215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.486433 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f663e38a-cc1c-4b21-a53a-c40b2ace16c5-hosts-file\") pod \"node-resolver-ztprt\" (UID: \"f663e38a-cc1c-4b21-a53a-c40b2ace16c5\") " pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.487215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.486511 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb02a1e3-ebbd-4628-a045-f883d47208a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.487215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.486678 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1295416-be49-4b38-9602-0c28e9e90055-konnectivity-ca\") pod \"konnectivity-agent-msp7f\" (UID: \"b1295416-be49-4b38-9602-0c28e9e90055\") " pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:29.487215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.486857 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad5f621b-f27f-4abb-a5be-557e0bbceb86-serviceca\") pod \"node-ca-sxk2l\" (UID: \"ad5f621b-f27f-4abb-a5be-557e0bbceb86\") " pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.487215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.486920 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-tmp\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.487215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.487028 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b6d741e6-fdd4-442e-9940-2588e3ef9419-kubelet-config\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:29.487215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.487162 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-sysctl-conf\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.487215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.487156 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-os-release\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.487680 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.487410 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c75397dd-0fb0-4607-9645-ce375ecc785d-cni-binary-copy\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.487680 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.487532 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-host-run-ovn-kubernetes\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.487680 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.487601 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b6d741e6-fdd4-442e-9940-2588e3ef9419-dbus\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:29.487858 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.487700 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-lib-modules\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.487858 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.487756 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-conf-dir\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.487858 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.487808 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-host-run-k8s-cni-cncf-io\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.488005 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.487924 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb02a1e3-ebbd-4628-a045-f883d47208a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.488085 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.488065 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f663e38a-cc1c-4b21-a53a-c40b2ace16c5-tmp-dir\") pod \"node-resolver-ztprt\" (UID: \"f663e38a-cc1c-4b21-a53a-c40b2ace16c5\") " pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.488144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.488100 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fd93e728-989f-4151-934f-ae52670e5531-ovnkube-script-lib\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.488209 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.488188 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-run-openvswitch\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.488336 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.488316 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c75397dd-0fb0-4607-9645-ce375ecc785d-system-cni-dir\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.488398 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.488383 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad5f621b-f27f-4abb-a5be-557e0bbceb86-host\") pod \"node-ca-sxk2l\" (UID: \"ad5f621b-f27f-4abb-a5be-557e0bbceb86\") " pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.488461 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.488447 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd93e728-989f-4151-934f-ae52670e5531-etc-openvswitch\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.488535 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.488514 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c75397dd-0fb0-4607-9645-ce375ecc785d-multus-daemon-config\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.489659 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.489632 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-etc-tuned\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.491301 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.490331 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fd93e728-989f-4151-934f-ae52670e5531-ovnkube-config\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.492661 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.492639 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:29.492759 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.492669 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:29.492759 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.492684 2583 projected.go:196] Error preparing data for projected volume kube-api-access-llzn5 for pod openshift-network-diagnostics/network-check-target-f8xn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:29.492873 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.492772 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5 podName:beb19682-944b-47da-b871-27e8f6dd449d nodeName:}" failed. No retries permitted until 2026-07-13 07:02:29.992727912 +0000 UTC m=+3.167231716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-llzn5" (UniqueName: "kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5") pod "network-check-target-f8xn4" (UID: "beb19682-944b-47da-b871-27e8f6dd449d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:29.496534 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.496511 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrbmz\" (UniqueName: \"kubernetes.io/projected/b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126-kube-api-access-hrbmz\") pod \"tuned-hdnpv\" (UID: \"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126\") " pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.498879 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.498828 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2lt7\" (UniqueName: \"kubernetes.io/projected/c75397dd-0fb0-4607-9645-ce375ecc785d-kube-api-access-h2lt7\") pod \"multus-jkjh6\" (UID: \"c75397dd-0fb0-4607-9645-ce375ecc785d\") " pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.499694 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.499667 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6slnd\" (UniqueName: \"kubernetes.io/projected/fb02a1e3-ebbd-4628-a045-f883d47208a6-kube-api-access-6slnd\") pod \"multus-additional-cni-plugins-l8fxn\" (UID: \"fb02a1e3-ebbd-4628-a045-f883d47208a6\") " pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.501331 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.500554 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxgd4\" (UniqueName: \"kubernetes.io/projected/ad5f621b-f27f-4abb-a5be-557e0bbceb86-kube-api-access-lxgd4\") pod \"node-ca-sxk2l\" (UID: \"ad5f621b-f27f-4abb-a5be-557e0bbceb86\") " pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.501331 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.500977 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6cv2\" (UniqueName: \"kubernetes.io/projected/f0296e96-219b-41af-8adb-e7d65fbf5bdb-kube-api-access-d6cv2\") pod \"iptables-alerter-4hqg6\" (UID: \"f0296e96-219b-41af-8adb-e7d65fbf5bdb\") " pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.501331 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.501010 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg4v2\" (UniqueName: \"kubernetes.io/projected/f663e38a-cc1c-4b21-a53a-c40b2ace16c5-kube-api-access-dg4v2\") pod \"node-resolver-ztprt\" (UID: \"f663e38a-cc1c-4b21-a53a-c40b2ace16c5\") " pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.502132 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.502085 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7jzl\" (UniqueName: \"kubernetes.io/projected/fd93e728-989f-4151-934f-ae52670e5531-kube-api-access-c7jzl\") pod \"ovnkube-node-lvl52\" (UID: \"fd93e728-989f-4151-934f-ae52670e5531\") " pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.502338 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.502294 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4kzt\" (UniqueName: \"kubernetes.io/projected/a68e2b3a-fc60-4be7-9bf1-664317d72b94-kube-api-access-h4kzt\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:29.502804 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.502761 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqkpr\" (UniqueName: \"kubernetes.io/projected/6763209a-3710-43fc-9ee8-a5a0caa7c5c5-kube-api-access-pqkpr\") pod \"aws-ebs-csi-driver-node-8dv9q\" (UID: \"6763209a-3710-43fc-9ee8-a5a0caa7c5c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.563324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.563300 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:29.571616 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.571596 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ztprt" Jul 13 07:02:29.572396 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:29.572372 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1295416_be49_4b38_9602_0c28e9e90055.slice/crio-1cd5fad7dfa910af9d421015b2af0a257c78f8ec26284a5058b7b2ed12c81956 WatchSource:0}: Error finding container 1cd5fad7dfa910af9d421015b2af0a257c78f8ec26284a5058b7b2ed12c81956: Status 404 returned error can't find the container with id 1cd5fad7dfa910af9d421015b2af0a257c78f8ec26284a5058b7b2ed12c81956 Jul 13 07:02:29.582099 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:29.582068 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf663e38a_cc1c_4b21_a53a_c40b2ace16c5.slice/crio-b979008cbf39029ae52ecb9cf67a02250bbc5c505267c1a0643b6492ad8c772f WatchSource:0}: Error finding container b979008cbf39029ae52ecb9cf67a02250bbc5c505267c1a0643b6492ad8c772f: Status 404 returned error can't find the container with id b979008cbf39029ae52ecb9cf67a02250bbc5c505267c1a0643b6492ad8c772f Jul 13 07:02:29.582763 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.582738 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sxk2l" Jul 13 07:02:29.590661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.590638 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jkjh6" Jul 13 07:02:29.591088 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:29.591062 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad5f621b_f27f_4abb_a5be_557e0bbceb86.slice/crio-e16b0968de752e5e09d1a5477f271b8f43c05d58b07b14c56e29754d4dabbca8 WatchSource:0}: Error finding container e16b0968de752e5e09d1a5477f271b8f43c05d58b07b14c56e29754d4dabbca8: Status 404 returned error can't find the container with id e16b0968de752e5e09d1a5477f271b8f43c05d58b07b14c56e29754d4dabbca8 Jul 13 07:02:29.599938 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:29.599909 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc75397dd_0fb0_4607_9645_ce375ecc785d.slice/crio-589dec6dc3d72e4a6479c79fd04a3be3b32e34c51f0ba2ac95062434010bd96c WatchSource:0}: Error finding container 589dec6dc3d72e4a6479c79fd04a3be3b32e34c51f0ba2ac95062434010bd96c: Status 404 returned error can't find the container with id 589dec6dc3d72e4a6479c79fd04a3be3b32e34c51f0ba2ac95062434010bd96c Jul 13 07:02:29.601832 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.601818 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" Jul 13 07:02:29.610961 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:29.610940 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb02a1e3_ebbd_4628_a045_f883d47208a6.slice/crio-bb5dd4c63433f293ea37fa2e1ac7d4a6c751a90d2ced21e34f6f639b3ed6f72a WatchSource:0}: Error finding container bb5dd4c63433f293ea37fa2e1ac7d4a6c751a90d2ced21e34f6f639b3ed6f72a: Status 404 returned error can't find the container with id bb5dd4c63433f293ea37fa2e1ac7d4a6c751a90d2ced21e34f6f639b3ed6f72a Jul 13 07:02:29.613215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.613194 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" Jul 13 07:02:29.620577 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:29.620558 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f5a6dc_4cbb_4e03_9b0e_b8c3f90de126.slice/crio-9a26c130e3bfadfbdd1d59723df52c872c1fd33d49a36b97554d678498b4cefc WatchSource:0}: Error finding container 9a26c130e3bfadfbdd1d59723df52c872c1fd33d49a36b97554d678498b4cefc: Status 404 returned error can't find the container with id 9a26c130e3bfadfbdd1d59723df52c872c1fd33d49a36b97554d678498b4cefc Jul 13 07:02:29.623865 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.623843 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" Jul 13 07:02:29.631343 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:29.631314 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6763209a_3710_43fc_9ee8_a5a0caa7c5c5.slice/crio-ccb001821b8416818f950c632534f41a14ab6edd9acbbe6d60169ab08f5c145e WatchSource:0}: Error finding container ccb001821b8416818f950c632534f41a14ab6edd9acbbe6d60169ab08f5c145e: Status 404 returned error can't find the container with id ccb001821b8416818f950c632534f41a14ab6edd9acbbe6d60169ab08f5c145e Jul 13 07:02:29.633052 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.633034 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4hqg6" Jul 13 07:02:29.642162 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.641838 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:29.642162 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:29.642107 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0296e96_219b_41af_8adb_e7d65fbf5bdb.slice/crio-412570c80c69f1df927c67448bce9d473ff789db4c53a0753982d7ff984fb615 WatchSource:0}: Error finding container 412570c80c69f1df927c67448bce9d473ff789db4c53a0753982d7ff984fb615: Status 404 returned error can't find the container with id 412570c80c69f1df927c67448bce9d473ff789db4c53a0753982d7ff984fb615 Jul 13 07:02:29.651832 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:02:29.651802 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd93e728_989f_4151_934f_ae52670e5531.slice/crio-4887b76f751deda3ed91877d6cdb7382ae09fbadcec1913673734719d9ab24fd WatchSource:0}: Error finding container 4887b76f751deda3ed91877d6cdb7382ae09fbadcec1913673734719d9ab24fd: Status 404 returned error can't find the container with id 4887b76f751deda3ed91877d6cdb7382ae09fbadcec1913673734719d9ab24fd Jul 13 07:02:29.668606 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.668583 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:29.668847 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.668829 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:29.986635 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.986596 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:29.986806 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:29.986644 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:29.986806 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.986751 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:29.986806 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.986778 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:29.986995 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.986834 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs podName:a68e2b3a-fc60-4be7-9bf1-664317d72b94 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:30.986809633 +0000 UTC m=+4.161313430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs") pod "network-metrics-daemon-jzrkj" (UID: "a68e2b3a-fc60-4be7-9bf1-664317d72b94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:29.986995 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:29.986857 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret podName:b6d741e6-fdd4-442e-9940-2588e3ef9419 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:30.986846344 +0000 UTC m=+4.161350130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret") pod "global-pull-secret-syncer-b2qc9" (UID: "b6d741e6-fdd4-442e-9940-2588e3ef9419") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:30.087582 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.087551 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzn5\" (UniqueName: \"kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5\") pod \"network-check-target-f8xn4\" (UID: \"beb19682-944b-47da-b871-27e8f6dd449d\") " pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:30.087735 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:30.087720 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:30.087775 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:30.087742 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:30.087775 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:30.087752 2583 projected.go:196] Error preparing data for projected volume kube-api-access-llzn5 for pod openshift-network-diagnostics/network-check-target-f8xn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:30.087870 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:30.087806 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5 podName:beb19682-944b-47da-b871-27e8f6dd449d nodeName:}" failed. No retries permitted until 2026-07-13 07:02:31.087792267 +0000 UTC m=+4.262296054 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-llzn5" (UniqueName: "kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5") pod "network-check-target-f8xn4" (UID: "beb19682-944b-47da-b871-27e8f6dd449d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:30.289477 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.289377 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 06:57:29 +0000 UTC" deadline="2028-03-28 19:42:14.392351144 +0000 UTC" Jul 13 07:02:30.289477 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.289416 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14988h39m44.102939491s" Jul 13 07:02:30.436244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.436171 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-msp7f" event={"ID":"b1295416-be49-4b38-9602-0c28e9e90055","Type":"ContainerStarted","Data":"1cd5fad7dfa910af9d421015b2af0a257c78f8ec26284a5058b7b2ed12c81956"} Jul 13 07:02:30.437877 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.437843 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sxk2l" event={"ID":"ad5f621b-f27f-4abb-a5be-557e0bbceb86","Type":"ContainerStarted","Data":"e16b0968de752e5e09d1a5477f271b8f43c05d58b07b14c56e29754d4dabbca8"} Jul 13 07:02:30.439870 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.439837 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ztprt" event={"ID":"f663e38a-cc1c-4b21-a53a-c40b2ace16c5","Type":"ContainerStarted","Data":"b979008cbf39029ae52ecb9cf67a02250bbc5c505267c1a0643b6492ad8c772f"} Jul 13 07:02:30.441764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.441734 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" event={"ID":"fd93e728-989f-4151-934f-ae52670e5531","Type":"ContainerStarted","Data":"4887b76f751deda3ed91877d6cdb7382ae09fbadcec1913673734719d9ab24fd"} Jul 13 07:02:30.443371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.443339 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4hqg6" event={"ID":"f0296e96-219b-41af-8adb-e7d65fbf5bdb","Type":"ContainerStarted","Data":"412570c80c69f1df927c67448bce9d473ff789db4c53a0753982d7ff984fb615"} Jul 13 07:02:30.446781 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.446753 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" event={"ID":"6763209a-3710-43fc-9ee8-a5a0caa7c5c5","Type":"ContainerStarted","Data":"ccb001821b8416818f950c632534f41a14ab6edd9acbbe6d60169ab08f5c145e"} Jul 13 07:02:30.449530 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.449509 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" event={"ID":"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126","Type":"ContainerStarted","Data":"9a26c130e3bfadfbdd1d59723df52c872c1fd33d49a36b97554d678498b4cefc"} Jul 13 07:02:30.450993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.450965 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" event={"ID":"fb02a1e3-ebbd-4628-a045-f883d47208a6","Type":"ContainerStarted","Data":"bb5dd4c63433f293ea37fa2e1ac7d4a6c751a90d2ced21e34f6f639b3ed6f72a"} Jul 13 07:02:30.452495 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.452469 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jkjh6" event={"ID":"c75397dd-0fb0-4607-9645-ce375ecc785d","Type":"ContainerStarted","Data":"589dec6dc3d72e4a6479c79fd04a3be3b32e34c51f0ba2ac95062434010bd96c"} Jul 13 07:02:30.995180 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.994610 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:30.995180 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:30.994669 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:30.995180 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:30.994792 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:30.995180 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:30.994793 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:30.995180 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:30.994853 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret podName:b6d741e6-fdd4-442e-9940-2588e3ef9419 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:32.994833464 +0000 UTC m=+6.169337259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret") pod "global-pull-secret-syncer-b2qc9" (UID: "b6d741e6-fdd4-442e-9940-2588e3ef9419") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:30.995180 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:30.994920 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs podName:a68e2b3a-fc60-4be7-9bf1-664317d72b94 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:32.994903428 +0000 UTC m=+6.169407223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs") pod "network-metrics-daemon-jzrkj" (UID: "a68e2b3a-fc60-4be7-9bf1-664317d72b94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:31.095738 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:31.095701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzn5\" (UniqueName: \"kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5\") pod \"network-check-target-f8xn4\" (UID: \"beb19682-944b-47da-b871-27e8f6dd449d\") " pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:31.095894 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:31.095860 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:31.095894 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:31.095880 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:31.095894 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:31.095892 2583 projected.go:196] Error preparing data for projected volume kube-api-access-llzn5 for pod openshift-network-diagnostics/network-check-target-f8xn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:31.096072 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:31.095958 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5 podName:beb19682-944b-47da-b871-27e8f6dd449d nodeName:}" failed. No retries permitted until 2026-07-13 07:02:33.095937587 +0000 UTC m=+6.270441389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-llzn5" (UniqueName: "kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5") pod "network-check-target-f8xn4" (UID: "beb19682-944b-47da-b871-27e8f6dd449d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:31.289889 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:31.289794 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 06:57:29 +0000 UTC" deadline="2028-03-01 13:30:25.900407808 +0000 UTC" Jul 13 07:02:31.289889 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:31.289847 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14334h27m54.610566726s" Jul 13 07:02:31.424255 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:31.423403 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:31.424255 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:31.423528 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:31.424255 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:31.423947 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:31.424255 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:31.424045 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:31.424255 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:31.424122 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:31.424255 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:31.424210 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:33.015779 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:33.015292 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:33.015779 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.015521 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:33.015779 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.015592 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret podName:b6d741e6-fdd4-442e-9940-2588e3ef9419 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:37.015572722 +0000 UTC m=+10.190076512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret") pod "global-pull-secret-syncer-b2qc9" (UID: "b6d741e6-fdd4-442e-9940-2588e3ef9419") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:33.015779 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:33.015619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:33.015779 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.015717 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:33.015779 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.015754 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs podName:a68e2b3a-fc60-4be7-9bf1-664317d72b94 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:37.015741918 +0000 UTC m=+10.190245710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs") pod "network-metrics-daemon-jzrkj" (UID: "a68e2b3a-fc60-4be7-9bf1-664317d72b94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:33.117335 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:33.116604 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzn5\" (UniqueName: \"kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5\") pod \"network-check-target-f8xn4\" (UID: \"beb19682-944b-47da-b871-27e8f6dd449d\") " pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:33.117335 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.116813 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:33.117335 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.116832 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:33.117335 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.116847 2583 projected.go:196] Error preparing data for projected volume kube-api-access-llzn5 for pod openshift-network-diagnostics/network-check-target-f8xn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:33.117335 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.116920 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5 podName:beb19682-944b-47da-b871-27e8f6dd449d nodeName:}" failed. No retries permitted until 2026-07-13 07:02:37.116901248 +0000 UTC m=+10.291405038 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-llzn5" (UniqueName: "kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5") pod "network-check-target-f8xn4" (UID: "beb19682-944b-47da-b871-27e8f6dd449d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:33.423925 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:33.423426 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:33.423925 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.423564 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:33.423925 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:33.423429 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:33.423925 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.423711 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:33.423925 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:33.423773 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:33.423925 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:33.423850 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:33.470299 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:33.470249 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" event={"ID":"cc3a3928a1786918c202ceb8745343f1","Type":"ContainerStarted","Data":"09f00d28014183d7c73c47749c6c7b445d6cb6982944cf8c62b5d36eb63cdc73"} Jul 13 07:02:34.476691 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:34.476650 2583 generic.go:356] "Generic (PLEG): container finished" podID="cc3a3928a1786918c202ceb8745343f1" containerID="09f00d28014183d7c73c47749c6c7b445d6cb6982944cf8c62b5d36eb63cdc73" exitCode=0 Jul 13 07:02:34.477183 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:34.476704 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" event={"ID":"cc3a3928a1786918c202ceb8745343f1","Type":"ContainerDied","Data":"09f00d28014183d7c73c47749c6c7b445d6cb6982944cf8c62b5d36eb63cdc73"} Jul 13 07:02:35.423214 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:35.423127 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:35.423390 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:35.423307 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:35.423390 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:35.423328 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:35.423622 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:35.423538 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:35.423992 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:35.423803 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:35.423992 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:35.423897 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:37.051666 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:37.051629 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:37.052138 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:37.051682 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:37.052138 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.051814 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:37.052138 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.051875 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret podName:b6d741e6-fdd4-442e-9940-2588e3ef9419 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:45.051856026 +0000 UTC m=+18.226359831 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret") pod "global-pull-secret-syncer-b2qc9" (UID: "b6d741e6-fdd4-442e-9940-2588e3ef9419") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:37.052355 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.052336 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:37.052412 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.052396 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs podName:a68e2b3a-fc60-4be7-9bf1-664317d72b94 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:45.052377425 +0000 UTC m=+18.226881234 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs") pod "network-metrics-daemon-jzrkj" (UID: "a68e2b3a-fc60-4be7-9bf1-664317d72b94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:37.152199 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:37.152163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzn5\" (UniqueName: \"kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5\") pod \"network-check-target-f8xn4\" (UID: \"beb19682-944b-47da-b871-27e8f6dd449d\") " pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:37.152429 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.152409 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:37.152501 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.152438 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:37.152501 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.152453 2583 projected.go:196] Error preparing data for projected volume kube-api-access-llzn5 for pod openshift-network-diagnostics/network-check-target-f8xn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:37.152606 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.152514 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5 podName:beb19682-944b-47da-b871-27e8f6dd449d nodeName:}" failed. No retries permitted until 2026-07-13 07:02:45.152494437 +0000 UTC m=+18.326998232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-llzn5" (UniqueName: "kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5") pod "network-check-target-f8xn4" (UID: "beb19682-944b-47da-b871-27e8f6dd449d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:37.424144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:37.423791 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:37.424144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:37.423821 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:37.424144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:37.423915 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:37.424144 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.424017 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:37.424495 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.424420 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:37.424534 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:37.424515 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:39.422446 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:39.422380 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:39.422951 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:39.422380 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:39.422951 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:39.422507 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:39.422951 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:39.422520 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:39.422951 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:39.422608 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:39.422951 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:39.422695 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:41.425154 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:41.425117 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:41.425604 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:41.425240 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:41.425604 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:41.425251 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:41.425604 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:41.425270 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:41.425604 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:41.425361 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:41.425604 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:41.425454 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:43.422755 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:43.422682 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:43.422755 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:43.422700 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:43.423291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:43.422696 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:43.423291 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:43.422808 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:43.423291 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:43.422877 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:43.423291 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:43.422969 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:45.113098 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:45.113065 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:45.113098 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:45.113104 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:45.113633 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.113221 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:45.113633 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.113231 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:45.113633 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.113275 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret podName:b6d741e6-fdd4-442e-9940-2588e3ef9419 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:01.113258275 +0000 UTC m=+34.287762064 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret") pod "global-pull-secret-syncer-b2qc9" (UID: "b6d741e6-fdd4-442e-9940-2588e3ef9419") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:45.113633 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.113289 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs podName:a68e2b3a-fc60-4be7-9bf1-664317d72b94 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:01.11328235 +0000 UTC m=+34.287786135 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs") pod "network-metrics-daemon-jzrkj" (UID: "a68e2b3a-fc60-4be7-9bf1-664317d72b94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:45.214342 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:45.214316 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzn5\" (UniqueName: \"kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5\") pod \"network-check-target-f8xn4\" (UID: \"beb19682-944b-47da-b871-27e8f6dd449d\") " pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:45.214486 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.214463 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:45.214486 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.214481 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:45.214590 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.214493 2583 projected.go:196] Error preparing data for projected volume kube-api-access-llzn5 for pod openshift-network-diagnostics/network-check-target-f8xn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:45.214590 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.214548 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5 podName:beb19682-944b-47da-b871-27e8f6dd449d nodeName:}" failed. No retries permitted until 2026-07-13 07:03:01.214532151 +0000 UTC m=+34.389035949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-llzn5" (UniqueName: "kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5") pod "network-check-target-f8xn4" (UID: "beb19682-944b-47da-b871-27e8f6dd449d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:45.425290 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:45.425198 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:45.425404 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:45.425199 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:45.425404 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.425321 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:45.425506 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.425397 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:45.425506 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:45.425199 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:45.425506 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:45.425488 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:47.423674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:47.423644 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:47.424084 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:47.423801 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:47.424084 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:47.423841 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:47.424084 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:47.423889 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:47.424084 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:47.423915 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:47.424084 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:47.423995 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:49.422598 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.422575 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:49.422943 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:49.422692 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:49.422943 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.422583 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:49.422943 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:49.422795 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:49.422943 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.422818 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:49.422943 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:49.422874 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:49.505850 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.505096 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal" event={"ID":"b573b3797697c17ef8fefe9494186a99","Type":"ContainerStarted","Data":"a3496d18124d84f270375d408fd5c19adbe34e8978d15a1ec16cbff8746bf9bc"} Jul 13 07:02:49.507611 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.507524 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" event={"ID":"fd93e728-989f-4151-934f-ae52670e5531","Type":"ContainerStarted","Data":"bee3fd045ba44f7283c5b66d752862510b9eda85af1c5fa3cde37c352cd7ad20"} Jul 13 07:02:49.509731 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.509705 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" event={"ID":"6763209a-3710-43fc-9ee8-a5a0caa7c5c5","Type":"ContainerStarted","Data":"af9fc68e63f14654d020db519132c6c3770f39423589fe2a592fbf8fefcd0e0a"} Jul 13 07:02:49.512165 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.512099 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" event={"ID":"b6f5a6dc-4cbb-4e03-9b0e-b8c3f90de126","Type":"ContainerStarted","Data":"25fd80af442afa24941afbcada49f16db3550a9f59cf67990b76015b2370784e"} Jul 13 07:02:49.514902 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.514861 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" event={"ID":"fb02a1e3-ebbd-4628-a045-f883d47208a6","Type":"ContainerStarted","Data":"ae4c12df8d9902633606a2c9e915131c9e680a1629c98cffca3cf6a897e96818"} Jul 13 07:02:49.517157 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.517135 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jkjh6" event={"ID":"c75397dd-0fb0-4607-9645-ce375ecc785d","Type":"ContainerStarted","Data":"e71f4286465aa2de3ae89273a61590236e517770ba3ab4555fb94f57895dbcf3"} Jul 13 07:02:49.520965 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.520604 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-msp7f" event={"ID":"b1295416-be49-4b38-9602-0c28e9e90055","Type":"ContainerStarted","Data":"3731ee55d81322622bab13ae975899f6c17ab9f65e0df457ab491713e28cda33"} Jul 13 07:02:49.522984 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.522961 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" event={"ID":"cc3a3928a1786918c202ceb8745343f1","Type":"ContainerStarted","Data":"69bf513a37aed66e02adebd9d7d821750c947d668502ddc61715b1e5556d6a3b"} Jul 13 07:02:49.524127 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.523895 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-57.ec2.internal" podStartSLOduration=20.523187586 podStartE2EDuration="20.523187586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:49.522338196 +0000 UTC m=+22.696842003" watchObservedRunningTime="2026-07-13 07:02:49.523187586 +0000 UTC m=+22.697691396" Jul 13 07:02:49.525191 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.524694 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sxk2l" event={"ID":"ad5f621b-f27f-4abb-a5be-557e0bbceb86","Type":"ContainerStarted","Data":"83bcc6ff09b9c64892d6d8484a820510b66b9d74ff9f46d03c17449bb0bdaf4f"} Jul 13 07:02:49.526893 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.526850 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ztprt" event={"ID":"f663e38a-cc1c-4b21-a53a-c40b2ace16c5","Type":"ContainerStarted","Data":"cf2b08cc5f0f67215487738da1aa4320d9ce9a555af30ce0c4e34ed77e5873e5"} Jul 13 07:02:49.556145 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.556003 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hdnpv" podStartSLOduration=1.937618846 podStartE2EDuration="21.555989868s" totalImagesPullingTime="19.618371022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:49.555859545 +0000 UTC m=+22.730363351" watchObservedRunningTime="2026-07-13 07:02:49.555989868 +0000 UTC m=+22.730493675" Jul 13 07:02:49.556439 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.556412 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jkjh6" podStartSLOduration=1.751171169 podStartE2EDuration="21.556404821s" totalImagesPullingTime="19.805233652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:49.540417524 +0000 UTC m=+22.714921333" watchObservedRunningTime="2026-07-13 07:02:49.556404821 +0000 UTC m=+22.730908628" Jul 13 07:02:49.569881 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.569836 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-msp7f" podStartSLOduration=10.129437748 podStartE2EDuration="21.569815951s" totalImagesPullingTime="11.440378203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:49.569240725 +0000 UTC m=+22.743744529" watchObservedRunningTime="2026-07-13 07:02:49.569815951 +0000 UTC m=+22.744319760" Jul 13 07:02:49.609838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.609801 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ztprt" podStartSLOduration=1.985386717 podStartE2EDuration="21.609777941s" totalImagesPullingTime="19.624391224s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:49.609352428 +0000 UTC m=+22.783856235" watchObservedRunningTime="2026-07-13 07:02:49.609777941 +0000 UTC m=+22.784281748" Jul 13 07:02:49.622670 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.622632 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-57.ec2.internal" podStartSLOduration=20.622615687 podStartE2EDuration="20.622615687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:49.622526587 +0000 UTC m=+22.797030396" watchObservedRunningTime="2026-07-13 07:02:49.622615687 +0000 UTC m=+22.797119494" Jul 13 07:02:49.640759 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.640721 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sxk2l" podStartSLOduration=10.219088002 podStartE2EDuration="21.640707355s" totalImagesPullingTime="11.421619353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:49.64035932 +0000 UTC m=+22.814863143" watchObservedRunningTime="2026-07-13 07:02:49.640707355 +0000 UTC m=+22.815211207" Jul 13 07:02:49.669357 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.669334 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:49.718843 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:49.718820 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:50.529998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.529963 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb02a1e3-ebbd-4628-a045-f883d47208a6" containerID="ae4c12df8d9902633606a2c9e915131c9e680a1629c98cffca3cf6a897e96818" exitCode=0 Jul 13 07:02:50.530670 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.530054 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" event={"ID":"fb02a1e3-ebbd-4628-a045-f883d47208a6","Type":"ContainerDied","Data":"ae4c12df8d9902633606a2c9e915131c9e680a1629c98cffca3cf6a897e96818"} Jul 13 07:02:50.533073 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.533051 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" event={"ID":"fd93e728-989f-4151-934f-ae52670e5531","Type":"ContainerStarted","Data":"aa98b434500b7dc0c06736f9355c9c25770b486610823b6e9f0dc1160c128244"} Jul 13 07:02:50.533157 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.533141 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" event={"ID":"fd93e728-989f-4151-934f-ae52670e5531","Type":"ContainerStarted","Data":"b8201da70dc3b0ad9f2275f1b0c1eb0968162e7b5c3fd74028e23aedb678ca33"} Jul 13 07:02:50.533231 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.533195 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" event={"ID":"fd93e728-989f-4151-934f-ae52670e5531","Type":"ContainerStarted","Data":"5add4488704e27bcf12d15ab1b0574116eefb2f609dbed17ab5097dab5b35a72"} Jul 13 07:02:50.533292 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.533262 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" event={"ID":"fd93e728-989f-4151-934f-ae52670e5531","Type":"ContainerStarted","Data":"458dcb786967cdd9f4446102822baec22c45c96e584b33d84a814379d8934ee4"} Jul 13 07:02:50.533344 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.533312 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" event={"ID":"fd93e728-989f-4151-934f-ae52670e5531","Type":"ContainerStarted","Data":"e2624ee7cb34a849f0da8a0ee6c4ecf0fc7966a2cffe7753d78eb7732298fab3"} Jul 13 07:02:50.534497 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.534471 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4hqg6" event={"ID":"f0296e96-219b-41af-8adb-e7d65fbf5bdb","Type":"ContainerStarted","Data":"7e827497c27af552bd2fcc5713d17f17807d2a7d8db958542e884bc6745c818e"} Jul 13 07:02:50.565327 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.565294 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4hqg6" podStartSLOduration=2.970706693 podStartE2EDuration="22.565281587s" totalImagesPullingTime="19.594574894s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:50.565227437 +0000 UTC m=+23.739731238" watchObservedRunningTime="2026-07-13 07:02:50.565281587 +0000 UTC m=+23.739785394" Jul 13 07:02:50.682457 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.682408 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:50.731788 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.731769 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:50.995332 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:50.995307 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 07:02:51.122427 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.122401 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:51.312901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.312809 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T07:02:50.995327251Z","UUID":"886aa030-ca0c-482a-9e1a-875afe3551d7","Handler":null,"Name":"","Endpoint":""} Jul 13 07:02:51.314394 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.314371 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 07:02:51.314498 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.314399 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 07:02:51.422822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.422788 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:51.422956 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.422790 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:51.422956 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:51.422918 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:51.422956 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.422795 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:51.423121 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:51.422988 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:51.423121 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:51.423083 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:51.472902 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.472877 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:51.524262 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.524237 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:51.539025 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.538957 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" event={"ID":"6763209a-3710-43fc-9ee8-a5a0caa7c5c5","Type":"ContainerStarted","Data":"4bc5235b111c0681b03a81e149c16c669f67900b489bd096f513122e35ebbfca"} Jul 13 07:02:51.932239 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:51.932191 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:52.008948 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.008919 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:52.063781 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.063759 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:52.185602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.184194 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:52.236184 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.236153 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:52.454677 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.454483 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:52.455060 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.455045 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:52.509625 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.509598 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:52.545024 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.544997 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" event={"ID":"fd93e728-989f-4151-934f-ae52670e5531","Type":"ContainerStarted","Data":"7bba246712c32a8d13289966d565055796d4e18e46c46dde765f383901e90f1a"} Jul 13 07:02:52.547305 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.547249 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" event={"ID":"6763209a-3710-43fc-9ee8-a5a0caa7c5c5","Type":"ContainerStarted","Data":"3b8c4b99dd30534ed836e0e666aa883aaca8584d4072986044d170718fa1dbce"} Jul 13 07:02:52.547456 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.547439 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:52.547882 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.547862 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-msp7f" Jul 13 07:02:52.566503 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.566457 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8dv9q" podStartSLOduration=2.318714519 podStartE2EDuration="24.566444572s" totalImagesPullingTime="22.247730053s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:52.565714524 +0000 UTC m=+25.740218331" watchObservedRunningTime="2026-07-13 07:02:52.566444572 +0000 UTC m=+25.740948378" Jul 13 07:02:52.620822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.620797 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:52.973797 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:52.973764 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ztprt_f663e38a-cc1c-4b21-a53a-c40b2ace16c5/dns-node-resolver/0.log" Jul 13 07:02:53.422950 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:53.422879 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:53.422950 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:53.422892 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:53.422950 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:53.422908 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:53.423139 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:53.422995 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:53.423139 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:53.423074 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:53.423224 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:53.423150 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:53.427157 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:53.427133 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:53.528762 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:53.528738 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:53.566684 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:53.566662 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sxk2l_ad5f621b-f27f-4abb-a5be-557e0bbceb86/node-ca/0.log" Jul 13 07:02:53.814215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:53.814150 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:53.926050 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:53.926025 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:54.690281 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:54.690246 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:55.422890 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:55.422818 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:55.423054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:55.422825 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:55.423054 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:55.422952 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:55.423054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:55.422835 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:55.423054 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:55.423024 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:55.423276 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:55.423109 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:56.558047 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:56.557867 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" event={"ID":"fd93e728-989f-4151-934f-ae52670e5531","Type":"ContainerStarted","Data":"63582fd1b0082dfcfb54e52cd883e8b7319069733da6e206e10c088d7b3203e0"} Jul 13 07:02:56.558636 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:56.558132 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:56.558636 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:56.558181 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:56.559723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:56.559696 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb02a1e3-ebbd-4628-a045-f883d47208a6" containerID="b02a73ce824abd20e7b88bf3aab53548b9d581f960aa37cee76f05fcc4743875" exitCode=0 Jul 13 07:02:56.559823 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:56.559737 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" event={"ID":"fb02a1e3-ebbd-4628-a045-f883d47208a6","Type":"ContainerDied","Data":"b02a73ce824abd20e7b88bf3aab53548b9d581f960aa37cee76f05fcc4743875"} Jul 13 07:02:56.572332 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:56.572313 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:56.586785 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:56.586738 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" podStartSLOduration=8.988684263 podStartE2EDuration="28.58672687s" totalImagesPullingTime="19.598042607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:56.586245104 +0000 UTC m=+29.760748930" watchObservedRunningTime="2026-07-13 07:02:56.58672687 +0000 UTC m=+29.761230677" Jul 13 07:02:57.423257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:57.423226 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:57.423505 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:57.423330 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:57.423505 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:57.423353 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:57.423505 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:57.423331 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:57.423505 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:57.423437 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:57.423505 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:57.423500 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:57.562246 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:57.562227 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:57.576188 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:57.576167 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:02:57.624874 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:57.624855 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:58.450657 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:58.450621 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:58.565342 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:58.565315 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb02a1e3-ebbd-4628-a045-f883d47208a6" containerID="1ea2a3151f68ec612163a157a36c1ea458e34783ffcb0620169a43bfa2fc2bb5" exitCode=0 Jul 13 07:02:58.565608 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:58.565395 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" event={"ID":"fb02a1e3-ebbd-4628-a045-f883d47208a6","Type":"ContainerDied","Data":"1ea2a3151f68ec612163a157a36c1ea458e34783ffcb0620169a43bfa2fc2bb5"} Jul 13 07:02:58.657231 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:58.657196 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:58.754451 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:58.754425 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:58.811777 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:58.811746 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:58.893237 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:58.893216 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:59.163364 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:59.163308 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:59.423008 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:59.422948 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:02:59.423008 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:59.422976 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:02:59.423231 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:59.422948 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:02:59.423231 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:59.423034 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:02:59.423231 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:59.423159 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:02:59.423330 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:02:59.423260 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:02:59.950130 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:02:59.950074 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:00.155607 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:00.155580 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:00.571655 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:00.571631 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb02a1e3-ebbd-4628-a045-f883d47208a6" containerID="850c5aab3357a3f79106abfbc150dbbc3f7b34c28a90b3d82e5f9dab16cf63ae" exitCode=0 Jul 13 07:03:00.571764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:00.571684 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" event={"ID":"fb02a1e3-ebbd-4628-a045-f883d47208a6","Type":"ContainerDied","Data":"850c5aab3357a3f79106abfbc150dbbc3f7b34c28a90b3d82e5f9dab16cf63ae"} Jul 13 07:03:00.746889 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:00.746861 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:00.847899 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:00.847848 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:01.129594 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:01.129515 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:01.129594 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:01.129559 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:01.130037 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.129663 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:03:01.130037 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.129662 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:03:01.130037 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.129712 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret podName:b6d741e6-fdd4-442e-9940-2588e3ef9419 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:33.129697777 +0000 UTC m=+66.304201562 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret") pod "global-pull-secret-syncer-b2qc9" (UID: "b6d741e6-fdd4-442e-9940-2588e3ef9419") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:03:01.130037 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.129756 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs podName:a68e2b3a-fc60-4be7-9bf1-664317d72b94 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:33.129740267 +0000 UTC m=+66.304244056 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs") pod "network-metrics-daemon-jzrkj" (UID: "a68e2b3a-fc60-4be7-9bf1-664317d72b94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:03:01.160948 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:01.160917 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:01.230269 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:01.230138 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzn5\" (UniqueName: \"kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5\") pod \"network-check-target-f8xn4\" (UID: \"beb19682-944b-47da-b871-27e8f6dd449d\") " pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:01.230438 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.230310 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:03:01.230438 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.230330 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:03:01.230438 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.230339 2583 projected.go:196] Error preparing data for projected volume kube-api-access-llzn5 for pod openshift-network-diagnostics/network-check-target-f8xn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:03:01.230438 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.230395 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5 podName:beb19682-944b-47da-b871-27e8f6dd449d nodeName:}" failed. No retries permitted until 2026-07-13 07:03:33.230377871 +0000 UTC m=+66.404881656 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-llzn5" (UniqueName: "kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5") pod "network-check-target-f8xn4" (UID: "beb19682-944b-47da-b871-27e8f6dd449d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:03:01.240855 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:01.240831 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:01.422607 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:01.422540 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:01.422718 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:01.422545 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:01.422718 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.422634 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:01.422718 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:01.422663 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:01.422850 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.422763 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:01.422890 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:01.422848 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:01.561468 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:01.561440 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:01.949800 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:01.949765 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:02.050863 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:02.050827 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:02.150654 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:02.150613 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:02.206765 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:02.206703 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:02.310814 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:02.310781 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:02.408086 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:02.408048 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:02.510818 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:02.510793 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:02.611338 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:02.611308 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:02.756235 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:02.756193 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:02.904191 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:02.904116 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:03.051096 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.051064 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:03.155764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.155687 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:03.207717 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.207692 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:03.299765 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.299736 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:03.400854 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.400822 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:03.422460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.422120 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:03.422460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.422120 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:03.422460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.422240 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:03.422460 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:03.422243 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:03.422460 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:03.422324 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:03.422460 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:03.422408 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:03.697717 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.697643 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:03.797734 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.797704 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:03.916983 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:03.916948 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:04.112051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:04.112019 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:04.293993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:04.293963 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:04.551437 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:04.551411 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:04.583812 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:04.583785 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" event={"ID":"fb02a1e3-ebbd-4628-a045-f883d47208a6","Type":"ContainerStarted","Data":"f2d8fa4d4e21f65a3c36f545b929a2205450930824fbe6d48c7a5e401035c205"} Jul 13 07:03:04.937449 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:04.937424 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:04.990698 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:04.990675 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:05.149188 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:05.149167 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:05.353190 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:05.353167 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:05.422405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:05.422387 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:05.422405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:05.422396 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:05.422540 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:05.422409 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:05.422540 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:05.422476 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:05.422647 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:05.422556 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:05.422713 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:05.422659 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:05.562417 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:05.562400 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:05.588159 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:05.588140 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb02a1e3-ebbd-4628-a045-f883d47208a6" containerID="f2d8fa4d4e21f65a3c36f545b929a2205450930824fbe6d48c7a5e401035c205" exitCode=0 Jul 13 07:03:05.588253 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:05.588176 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" event={"ID":"fb02a1e3-ebbd-4628-a045-f883d47208a6","Type":"ContainerDied","Data":"f2d8fa4d4e21f65a3c36f545b929a2205450930824fbe6d48c7a5e401035c205"} Jul 13 07:03:05.952671 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:05.952618 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:06.548392 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:06.548369 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:06.593585 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:06.593564 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb02a1e3-ebbd-4628-a045-f883d47208a6" containerID="cc7203b3d6bceeb35764107c6601159f876b691e67e544f70128112066045a75" exitCode=0 Jul 13 07:03:06.593678 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:06.593602 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" event={"ID":"fb02a1e3-ebbd-4628-a045-f883d47208a6","Type":"ContainerDied","Data":"cc7203b3d6bceeb35764107c6601159f876b691e67e544f70128112066045a75"} Jul 13 07:03:06.956584 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:06.956395 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:07.150602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:07.150578 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:07.355237 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:07.355214 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:07.422709 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:07.422689 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:07.422815 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:07.422750 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:07.422815 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:07.422780 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:07.422928 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:07.422897 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:07.422980 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:07.422965 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:07.423031 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:07.423017 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:07.535617 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:07.535601 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:07.598911 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:07.598890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" event={"ID":"fb02a1e3-ebbd-4628-a045-f883d47208a6","Type":"ContainerStarted","Data":"640d5ea20b0c4eaf88e67df765929c1a48b6aa4fa64416fa126cdffa342b53c0"} Jul 13 07:03:07.622229 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:07.622155 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l8fxn" podStartSLOduration=8.533124376 podStartE2EDuration="39.622142908s" totalImagesPullingTime="31.089018532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:07.620692316 +0000 UTC m=+40.795196122" watchObservedRunningTime="2026-07-13 07:03:07.622142908 +0000 UTC m=+40.796646714" Jul 13 07:03:07.751510 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:07.751489 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:08.157560 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:08.157534 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:08.748959 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:08.748936 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:09.153998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:09.153939 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:09.331061 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:09.331035 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:09.422096 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:09.422038 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:09.422096 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:09.422049 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:09.422096 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:09.422038 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:09.422310 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:09.422149 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:09.422310 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:09.422221 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:09.422310 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:09.422286 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:09.432045 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:09.432027 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:09.545903 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:09.545879 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:11.422969 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:11.422942 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:11.422969 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:11.422956 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:11.422969 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:11.422962 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:11.423491 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:11.423061 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:11.423491 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:11.423156 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:11.423491 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:11.423233 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:11.438822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:11.438799 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:11.493883 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:11.493862 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:13.422695 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:13.422665 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:13.423044 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:13.422669 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:13.423044 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:13.422751 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:13.423044 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:13.422777 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:13.423044 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:13.422822 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:13.423044 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:13.422898 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:14.538726 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:14.538698 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:14.593999 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:14.593970 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:15.422313 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:15.422280 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:15.422456 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:15.422376 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:15.422456 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:15.422277 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:15.422456 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:15.422410 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:15.422579 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:15.422454 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:15.422579 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:15.422501 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:16.204717 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:16.204687 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:16.755681 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:16.755655 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:17.065959 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.065873 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:17.165484 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.165457 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:17.422929 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.422861 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:17.423346 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.422930 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:17.423346 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:17.422945 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:17.423346 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:17.423024 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:17.423346 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.423061 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:17.423346 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:17.423153 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:17.997036 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.997011 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f8xn4"] Jul 13 07:03:17.997144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.997096 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:17.997188 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:17.997171 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:17.997532 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.997512 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b2qc9"] Jul 13 07:03:17.997592 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.997582 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:17.997665 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:17.997650 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:17.998853 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.998826 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jzrkj"] Jul 13 07:03:17.998959 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:17.998913 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:17.999036 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:17.999016 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:18.646500 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:18.646409 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:18.747095 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:18.747067 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:18.829190 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:18.829167 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:19.381373 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:19.381347 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:19.422020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:19.421997 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:19.422020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:19.422020 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:19.422165 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:19.422101 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:19.422165 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:19.422003 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:19.422251 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:19.422192 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:19.422307 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:19.422283 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:19.451625 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:19.451604 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:19.607893 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:19.607861 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:21.422962 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:21.422790 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:21.423438 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:21.422790 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:21.423438 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:21.423026 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2qc9" podUID="b6d741e6-fdd4-442e-9940-2588e3ef9419" Jul 13 07:03:21.423438 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:21.423103 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzrkj" podUID="a68e2b3a-fc60-4be7-9bf1-664317d72b94" Jul 13 07:03:21.423438 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:21.422796 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:21.423438 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:21.423167 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8xn4" podUID="beb19682-944b-47da-b871-27e8f6dd449d" Jul 13 07:03:22.148950 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.148888 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-57.ec2.internal" event="NodeReady" Jul 13 07:03:22.149065 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.148980 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 07:03:22.184006 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.183983 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:22.200144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.200125 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9ttvg"] Jul 13 07:03:22.202018 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.202002 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.204247 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.204199 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fpkv5"] Jul 13 07:03:22.205360 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.205335 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ts2w7\"" Jul 13 07:03:22.205468 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.205360 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 07:03:22.205468 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.205341 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 07:03:22.205887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.205874 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.208506 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.208482 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 07:03:22.208971 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.208957 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 07:03:22.209050 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.208989 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-krdct\"" Jul 13 07:03:22.209112 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.209046 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 07:03:22.209166 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.209135 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 07:03:22.219868 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.219811 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9ttvg"] Jul 13 07:03:22.220582 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.220565 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fpkv5"] Jul 13 07:03:22.283727 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.283705 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-crio-socket\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.283811 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.283737 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs99r\" (UniqueName: \"kubernetes.io/projected/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-kube-api-access-zs99r\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.283811 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.283760 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38de7579-9ff9-4d60-8d06-5534eb04507d-tmp-dir\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.283881 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.283836 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.283881 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.283864 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-data-volume\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.283941 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.283882 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.283941 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.283900 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38de7579-9ff9-4d60-8d06-5534eb04507d-metrics-tls\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.284003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.283950 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38de7579-9ff9-4d60-8d06-5534eb04507d-config-volume\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.284003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.283979 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghj9f\" (UniqueName: \"kubernetes.io/projected/38de7579-9ff9-4d60-8d06-5534eb04507d-kube-api-access-ghj9f\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.300029 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.300011 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4tcb9"] Jul 13 07:03:22.301648 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.301633 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4tcb9" Jul 13 07:03:22.304217 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.304152 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 07:03:22.304217 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.304188 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wcqht\"" Jul 13 07:03:22.304217 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.304161 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 07:03:22.304407 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.304226 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 07:03:22.309998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.309976 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4tcb9"] Jul 13 07:03:22.333594 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.333574 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:22.384239 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384198 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.384341 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384256 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38de7579-9ff9-4d60-8d06-5534eb04507d-metrics-tls\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.384341 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384287 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38de7579-9ff9-4d60-8d06-5534eb04507d-config-volume\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.384341 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384311 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghj9f\" (UniqueName: \"kubernetes.io/projected/38de7579-9ff9-4d60-8d06-5534eb04507d-kube-api-access-ghj9f\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.384489 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384359 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10443be9-db7c-47da-8a59-ee567dfa62c8-cert\") pod \"ingress-canary-4tcb9\" (UID: \"10443be9-db7c-47da-8a59-ee567dfa62c8\") " pod="openshift-ingress-canary/ingress-canary-4tcb9" Jul 13 07:03:22.384489 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384395 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-crio-socket\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.384489 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384424 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwhcs\" (UniqueName: \"kubernetes.io/projected/10443be9-db7c-47da-8a59-ee567dfa62c8-kube-api-access-zwhcs\") pod \"ingress-canary-4tcb9\" (UID: \"10443be9-db7c-47da-8a59-ee567dfa62c8\") " pod="openshift-ingress-canary/ingress-canary-4tcb9" Jul 13 07:03:22.384607 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384501 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs99r\" (UniqueName: \"kubernetes.io/projected/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-kube-api-access-zs99r\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.384607 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38de7579-9ff9-4d60-8d06-5534eb04507d-tmp-dir\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.384607 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384576 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.384607 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384585 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-crio-socket\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.384607 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384605 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-data-volume\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.384881 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384862 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38de7579-9ff9-4d60-8d06-5534eb04507d-tmp-dir\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.384956 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384937 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38de7579-9ff9-4d60-8d06-5534eb04507d-config-volume\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.384990 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.384953 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-data-volume\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.385128 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.385115 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.386654 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.386634 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:22.388074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.388053 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.388149 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.388081 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38de7579-9ff9-4d60-8d06-5534eb04507d-metrics-tls\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.391629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.391607 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs99r\" (UniqueName: \"kubernetes.io/projected/13b90926-ab1e-4e33-a60e-45d5ba7d5abb-kube-api-access-zs99r\") pod \"insights-runtime-extractor-fpkv5\" (UID: \"13b90926-ab1e-4e33-a60e-45d5ba7d5abb\") " pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.391723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.391669 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghj9f\" (UniqueName: \"kubernetes.io/projected/38de7579-9ff9-4d60-8d06-5534eb04507d-kube-api-access-ghj9f\") pod \"dns-default-9ttvg\" (UID: \"38de7579-9ff9-4d60-8d06-5534eb04507d\") " pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.485023 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.484996 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10443be9-db7c-47da-8a59-ee567dfa62c8-cert\") pod \"ingress-canary-4tcb9\" (UID: \"10443be9-db7c-47da-8a59-ee567dfa62c8\") " pod="openshift-ingress-canary/ingress-canary-4tcb9" Jul 13 07:03:22.485321 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.485037 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwhcs\" (UniqueName: \"kubernetes.io/projected/10443be9-db7c-47da-8a59-ee567dfa62c8-kube-api-access-zwhcs\") pod \"ingress-canary-4tcb9\" (UID: \"10443be9-db7c-47da-8a59-ee567dfa62c8\") " pod="openshift-ingress-canary/ingress-canary-4tcb9" Jul 13 07:03:22.486897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.486880 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10443be9-db7c-47da-8a59-ee567dfa62c8-cert\") pod \"ingress-canary-4tcb9\" (UID: \"10443be9-db7c-47da-8a59-ee567dfa62c8\") " pod="openshift-ingress-canary/ingress-canary-4tcb9" Jul 13 07:03:22.493440 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.493422 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwhcs\" (UniqueName: \"kubernetes.io/projected/10443be9-db7c-47da-8a59-ee567dfa62c8-kube-api-access-zwhcs\") pod \"ingress-canary-4tcb9\" (UID: \"10443be9-db7c-47da-8a59-ee567dfa62c8\") " pod="openshift-ingress-canary/ingress-canary-4tcb9" Jul 13 07:03:22.513304 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.513286 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:22.517662 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.517647 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fpkv5" Jul 13 07:03:22.537881 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.537845 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:22.609603 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.609556 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4tcb9" Jul 13 07:03:22.615053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.615031 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:22.657157 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.657126 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9ttvg"] Jul 13 07:03:22.661518 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.661490 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fpkv5"] Jul 13 07:03:22.664407 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:22.664377 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38de7579_9ff9_4d60_8d06_5534eb04507d.slice/crio-9ad48c14e7a4247af624554854a65d4566c42d81a2d5f9855eab76ffc2f518b7 WatchSource:0}: Error finding container 9ad48c14e7a4247af624554854a65d4566c42d81a2d5f9855eab76ffc2f518b7: Status 404 returned error can't find the container with id 9ad48c14e7a4247af624554854a65d4566c42d81a2d5f9855eab76ffc2f518b7 Jul 13 07:03:22.666079 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:22.666040 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b90926_ab1e_4e33_a60e_45d5ba7d5abb.slice/crio-332c326c3b10db3bc16ed9d2c0158bc8c7583f7aacaf52bf6cb69eca93e90a2c WatchSource:0}: Error finding container 332c326c3b10db3bc16ed9d2c0158bc8c7583f7aacaf52bf6cb69eca93e90a2c: Status 404 returned error can't find the container with id 332c326c3b10db3bc16ed9d2c0158bc8c7583f7aacaf52bf6cb69eca93e90a2c Jul 13 07:03:22.731195 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:22.731172 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4tcb9"] Jul 13 07:03:22.733848 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:22.733828 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10443be9_db7c_47da_8a59_ee567dfa62c8.slice/crio-b34609be7da5cc0a1aa447efa71b9f6d9d90ef85708551f31db481cac490a9e2 WatchSource:0}: Error finding container b34609be7da5cc0a1aa447efa71b9f6d9d90ef85708551f31db481cac490a9e2: Status 404 returned error can't find the container with id b34609be7da5cc0a1aa447efa71b9f6d9d90ef85708551f31db481cac490a9e2 Jul 13 07:03:23.423307 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.422580 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:23.423307 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.422614 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:23.423307 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.422580 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:23.426367 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.425484 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z8tdw\"" Jul 13 07:03:23.428083 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.426931 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 07:03:23.428083 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.427137 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fh46b\"" Jul 13 07:03:23.428083 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.427358 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 07:03:23.428083 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.427574 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 07:03:23.428083 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.427762 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 07:03:23.641567 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.641470 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4tcb9" event={"ID":"10443be9-db7c-47da-8a59-ee567dfa62c8","Type":"ContainerStarted","Data":"b34609be7da5cc0a1aa447efa71b9f6d9d90ef85708551f31db481cac490a9e2"} Jul 13 07:03:23.645400 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.645314 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fpkv5" event={"ID":"13b90926-ab1e-4e33-a60e-45d5ba7d5abb","Type":"ContainerStarted","Data":"332c326c3b10db3bc16ed9d2c0158bc8c7583f7aacaf52bf6cb69eca93e90a2c"} Jul 13 07:03:23.647830 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.647799 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9ttvg" event={"ID":"38de7579-9ff9-4d60-8d06-5534eb04507d","Type":"ContainerStarted","Data":"9ad48c14e7a4247af624554854a65d4566c42d81a2d5f9855eab76ffc2f518b7"} Jul 13 07:03:23.855602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.855450 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:23.956022 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:23.955988 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:25.341650 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.341624 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:25.443033 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.443001 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:25.818140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.818109 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:25.862781 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.862742 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-zrg6c"] Jul 13 07:03:25.876994 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.876965 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:25.878139 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.878120 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-zrg6c"] Jul 13 07:03:25.878273 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.878242 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:25.880913 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.880781 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 07:03:25.880913 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.880791 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 07:03:25.880913 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.880812 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 07:03:25.881992 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.881970 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 07:03:25.882102 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.882064 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 07:03:25.882163 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.882137 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2nphr\"" Jul 13 07:03:25.932774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:25.932753 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:26.014008 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.013983 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-metrics-client-ca\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.014140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.014017 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.014140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.014049 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.014140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.014103 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnc9w\" (UniqueName: \"kubernetes.io/projected/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-kube-api-access-vnc9w\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.115117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.115053 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.115117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.115097 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.115303 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.115149 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnc9w\" (UniqueName: \"kubernetes.io/projected/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-kube-api-access-vnc9w\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.115303 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.115233 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-metrics-client-ca\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.115993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.115967 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-metrics-client-ca\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.118272 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.118246 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.118683 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.118667 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.124124 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.124029 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnc9w\" (UniqueName: \"kubernetes.io/projected/9cb1c18f-81e6-4761-85cc-c65dc0774aaa-kube-api-access-vnc9w\") pod \"prometheus-operator-64785b6675-zrg6c\" (UID: \"9cb1c18f-81e6-4761-85cc-c65dc0774aaa\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.189035 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.189002 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" Jul 13 07:03:26.741897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.741864 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:26.802006 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:26.801971 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:28.142941 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.142911 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:28.193016 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.192991 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:28.202247 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.202223 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-zrg6c"] Jul 13 07:03:28.205007 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:28.204982 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb1c18f_81e6_4761_85cc_c65dc0774aaa.slice/crio-33770a5cc188dc11cdf772071f8fcc9d3c6b383ddae600f80af3d3970564cbc1 WatchSource:0}: Error finding container 33770a5cc188dc11cdf772071f8fcc9d3c6b383ddae600f80af3d3970564cbc1: Status 404 returned error can't find the container with id 33770a5cc188dc11cdf772071f8fcc9d3c6b383ddae600f80af3d3970564cbc1 Jul 13 07:03:28.660960 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.660874 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" event={"ID":"9cb1c18f-81e6-4761-85cc-c65dc0774aaa","Type":"ContainerStarted","Data":"33770a5cc188dc11cdf772071f8fcc9d3c6b383ddae600f80af3d3970564cbc1"} Jul 13 07:03:28.662349 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.662322 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4tcb9" event={"ID":"10443be9-db7c-47da-8a59-ee567dfa62c8","Type":"ContainerStarted","Data":"674ab36968ddb342a533a2a409bbfbbe0d56f6016b61dce722235cb050bbd53a"} Jul 13 07:03:28.663709 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.663688 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fpkv5" event={"ID":"13b90926-ab1e-4e33-a60e-45d5ba7d5abb","Type":"ContainerStarted","Data":"a001ad39739818f4b26f96341bcbd7cbe9ddf23cbf8926687ecca3358c110b3b"} Jul 13 07:03:28.665332 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.665310 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9ttvg" event={"ID":"38de7579-9ff9-4d60-8d06-5534eb04507d","Type":"ContainerStarted","Data":"89dd4e39a0608f60380e4b3903acb564fedfd9ef2a48ef426a38aba2f6e105ff"} Jul 13 07:03:28.665410 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.665350 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9ttvg" event={"ID":"38de7579-9ff9-4d60-8d06-5534eb04507d","Type":"ContainerStarted","Data":"06971874863c641bf0747be0f00bdb395bb1843de902289f6868c4ffbb7d1ccf"} Jul 13 07:03:28.665410 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.665370 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:28.682404 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.682365 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4tcb9" podStartSLOduration=1.331514019 podStartE2EDuration="6.682348291s" totalImagesPullingTime="5.350834272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:28.6822385 +0000 UTC m=+61.856742307" watchObservedRunningTime="2026-07-13 07:03:28.682348291 +0000 UTC m=+61.856852100" Jul 13 07:03:28.714344 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:28.714299 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9ttvg" podStartSLOduration=1.3025226650000001 podStartE2EDuration="6.714280514s" totalImagesPullingTime="5.411757849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:28.713926376 +0000 UTC m=+61.888430186" watchObservedRunningTime="2026-07-13 07:03:28.714280514 +0000 UTC m=+61.888784325" Jul 13 07:03:29.578567 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:29.578540 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lvl52" Jul 13 07:03:29.674588 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:29.674540 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" event={"ID":"9cb1c18f-81e6-4761-85cc-c65dc0774aaa","Type":"ContainerStarted","Data":"e45bd87daa8604c8a1e6686174ecdc7a3a0c98e5b4a770fd61fce1d897f07634"} Jul 13 07:03:29.674766 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:29.674605 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" event={"ID":"9cb1c18f-81e6-4761-85cc-c65dc0774aaa","Type":"ContainerStarted","Data":"a2ce4aa23a1a8919b5cc818518efcd68160f5cec4ee652620ed6c0908f9d79a5"} Jul 13 07:03:29.677843 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:29.676886 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fpkv5" event={"ID":"13b90926-ab1e-4e33-a60e-45d5ba7d5abb","Type":"ContainerStarted","Data":"7d5437bb5fd8a2dcb33a18966cf51b85f29e8fc0c4dc9e4cfd60fee473134c48"} Jul 13 07:03:29.691582 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:29.691540 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-zrg6c" podStartSLOduration=3.414573962 podStartE2EDuration="4.691526643s" totalImagesPullingTime="1.276952681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:29.690869197 +0000 UTC m=+62.865373027" watchObservedRunningTime="2026-07-13 07:03:29.691526643 +0000 UTC m=+62.866030450" Jul 13 07:03:29.713172 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:29.713148 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:29.814556 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:29.814483 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:30.969128 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:30.969090 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:31.197243 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.197145 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:31.219931 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.219900 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4"] Jul 13 07:03:31.224831 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.224801 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.227851 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.227824 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 07:03:31.227988 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.227851 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 07:03:31.227988 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.227950 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f5mhr\"" Jul 13 07:03:31.236128 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.236103 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4"] Jul 13 07:03:31.241403 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.241380 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8lltp"] Jul 13 07:03:31.244132 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.244112 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.246255 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.246189 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-6cvmw"] Jul 13 07:03:31.248284 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.248268 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 07:03:31.248284 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.248278 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 07:03:31.248444 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.248401 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.248533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.248511 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zxnwr\"" Jul 13 07:03:31.248594 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.248528 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 07:03:31.251694 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.251674 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4kplc\"" Jul 13 07:03:31.252025 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.251990 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 13 07:03:31.252099 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.252073 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 13 07:03:31.252381 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.252230 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 13 07:03:31.264112 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.264091 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-6cvmw"] Jul 13 07:03:31.302397 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.302370 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:31.358507 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358481 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f47ad689-c9b0-4f6f-9871-adec98f855fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.358624 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358517 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-wtmp\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.358624 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358540 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.358624 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-textfile\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.358790 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358621 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-tls\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.358790 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358662 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b2a8f04-ea30-4461-a460-7fb3a891e063-metrics-client-ca\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.358790 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358713 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/618475e3-496a-4bdc-bea8-8222dc14a179-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.358790 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358736 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b2a8f04-ea30-4461-a460-7fb3a891e063-root\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.358985 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358787 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.358985 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358826 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.358985 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358856 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/618475e3-496a-4bdc-bea8-8222dc14a179-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.358985 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358938 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f47ad689-c9b0-4f6f-9871-adec98f855fe-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.358985 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.358975 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nszv\" (UniqueName: \"kubernetes.io/projected/f47ad689-c9b0-4f6f-9871-adec98f855fe-kube-api-access-6nszv\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.359258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.359005 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.359258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.359039 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f47ad689-c9b0-4f6f-9871-adec98f855fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.359258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.359067 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn8z5\" (UniqueName: \"kubernetes.io/projected/618475e3-496a-4bdc-bea8-8222dc14a179-kube-api-access-xn8z5\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.359258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.359098 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-accelerators-collector-config\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.359258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.359157 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b2a8f04-ea30-4461-a460-7fb3a891e063-sys\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.359258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.359182 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndp6w\" (UniqueName: \"kubernetes.io/projected/6b2a8f04-ea30-4461-a460-7fb3a891e063-kube-api-access-ndp6w\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.373439 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.373419 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:31.459553 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459526 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-accelerators-collector-config\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.459682 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459564 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b2a8f04-ea30-4461-a460-7fb3a891e063-sys\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.459682 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459580 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndp6w\" (UniqueName: \"kubernetes.io/projected/6b2a8f04-ea30-4461-a460-7fb3a891e063-kube-api-access-ndp6w\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.459682 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459657 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b2a8f04-ea30-4461-a460-7fb3a891e063-sys\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.459844 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459786 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f47ad689-c9b0-4f6f-9871-adec98f855fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.459844 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459830 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-wtmp\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.459949 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.459949 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459890 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-textfile\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.459949 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459918 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-tls\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.459949 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459946 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b2a8f04-ea30-4461-a460-7fb3a891e063-metrics-client-ca\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.460140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.459989 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/618475e3-496a-4bdc-bea8-8222dc14a179-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.460140 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:31.460014 2583 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 13 07:03:31.460140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460026 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-wtmp\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.460140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460070 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b2a8f04-ea30-4461-a460-7fb3a891e063-root\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.460140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b2a8f04-ea30-4461-a460-7fb3a891e063-root\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.460140 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:31.460077 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-tls podName:618475e3-496a-4bdc-bea8-8222dc14a179 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:31.960058304 +0000 UTC m=+65.134562093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-6cvmw" (UID: "618475e3-496a-4bdc-bea8-8222dc14a179") : secret "kube-state-metrics-tls" not found Jul 13 07:03:31.460460 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:31.460084 2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 13 07:03:31.460460 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:31.460292 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-tls podName:6b2a8f04-ea30-4461-a460-7fb3a891e063 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:31.960270431 +0000 UTC m=+65.134774219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-tls") pod "node-exporter-8lltp" (UID: "6b2a8f04-ea30-4461-a460-7fb3a891e063") : secret "node-exporter-tls" not found Jul 13 07:03:31.460460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460300 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-accelerators-collector-config\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.460460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460361 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-textfile\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.460460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460371 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.460460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460418 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.460460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460447 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/618475e3-496a-4bdc-bea8-8222dc14a179-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.460807 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460496 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f47ad689-c9b0-4f6f-9871-adec98f855fe-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.460807 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460522 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nszv\" (UniqueName: \"kubernetes.io/projected/f47ad689-c9b0-4f6f-9871-adec98f855fe-kube-api-access-6nszv\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.460807 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460534 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b2a8f04-ea30-4461-a460-7fb3a891e063-metrics-client-ca\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.460807 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460550 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.460807 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460587 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f47ad689-c9b0-4f6f-9871-adec98f855fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.460807 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460616 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn8z5\" (UniqueName: \"kubernetes.io/projected/618475e3-496a-4bdc-bea8-8222dc14a179-kube-api-access-xn8z5\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.461139 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.460841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/618475e3-496a-4bdc-bea8-8222dc14a179-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.461139 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:31.460945 2583 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 13 07:03:31.461139 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:03:31.461002 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f47ad689-c9b0-4f6f-9871-adec98f855fe-openshift-state-metrics-tls podName:f47ad689-c9b0-4f6f-9871-adec98f855fe nodeName:}" failed. No retries permitted until 2026-07-13 07:03:31.960987005 +0000 UTC m=+65.135490797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f47ad689-c9b0-4f6f-9871-adec98f855fe-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-t8lb4" (UID: "f47ad689-c9b0-4f6f-9871-adec98f855fe") : secret "openshift-state-metrics-tls" not found Jul 13 07:03:31.461350 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.461139 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/618475e3-496a-4bdc-bea8-8222dc14a179-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.461350 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.461184 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f47ad689-c9b0-4f6f-9871-adec98f855fe-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.461484 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.461469 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.463091 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.462958 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f47ad689-c9b0-4f6f-9871-adec98f855fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.463091 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.463033 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.463341 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.463317 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.471491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.471352 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndp6w\" (UniqueName: \"kubernetes.io/projected/6b2a8f04-ea30-4461-a460-7fb3a891e063-kube-api-access-ndp6w\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.472273 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.472247 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nszv\" (UniqueName: \"kubernetes.io/projected/f47ad689-c9b0-4f6f-9871-adec98f855fe-kube-api-access-6nszv\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.472527 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.472506 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn8z5\" (UniqueName: \"kubernetes.io/projected/618475e3-496a-4bdc-bea8-8222dc14a179-kube-api-access-xn8z5\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.621167 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.621124 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:31.724936 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.724866 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:31.822058 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.822025 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:31.921866 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.921836 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:31.964335 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.964314 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f47ad689-c9b0-4f6f-9871-adec98f855fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.964436 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.964364 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.964436 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.964395 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-tls\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:31.966833 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.966770 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/618475e3-496a-4bdc-bea8-8222dc14a179-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6cvmw\" (UID: \"618475e3-496a-4bdc-bea8-8222dc14a179\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:31.967394 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.967348 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f47ad689-c9b0-4f6f-9871-adec98f855fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-t8lb4\" (UID: \"f47ad689-c9b0-4f6f-9871-adec98f855fe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:31.967678 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:31.967659 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b2a8f04-ea30-4461-a460-7fb3a891e063-node-exporter-tls\") pod \"node-exporter-8lltp\" (UID: \"6b2a8f04-ea30-4461-a460-7fb3a891e063\") " pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:32.113631 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.113585 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:32.135526 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.135508 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" Jul 13 07:03:32.155367 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.155349 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8lltp" Jul 13 07:03:32.161601 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.161591 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" Jul 13 07:03:32.163995 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:32.163973 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b2a8f04_ea30_4461_a460_7fb3a891e063.slice/crio-449a613337a38c1fe92b02fe0463c58a904d047171d1873ca9000121bb2b64f9 WatchSource:0}: Error finding container 449a613337a38c1fe92b02fe0463c58a904d047171d1873ca9000121bb2b64f9: Status 404 returned error can't find the container with id 449a613337a38c1fe92b02fe0463c58a904d047171d1873ca9000121bb2b64f9 Jul 13 07:03:32.210365 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.210324 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:32.270616 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.270586 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4"] Jul 13 07:03:32.272412 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.272373 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:32.273593 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:32.273570 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47ad689_c9b0_4f6f_9871_adec98f855fe.slice/crio-d1eca70b9e3a2c496e05994dada18715cba00f3990c3258758d4c70a5e795b2b WatchSource:0}: Error finding container d1eca70b9e3a2c496e05994dada18715cba00f3990c3258758d4c70a5e795b2b: Status 404 returned error can't find the container with id d1eca70b9e3a2c496e05994dada18715cba00f3990c3258758d4c70a5e795b2b Jul 13 07:03:32.296722 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.296705 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-6cvmw"] Jul 13 07:03:32.301111 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:32.301089 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod618475e3_496a_4bdc_bea8_8222dc14a179.slice/crio-66292d17adec984affcdfb1618de89a5ddf4717cdf945cbadc059da67056ea73 WatchSource:0}: Error finding container 66292d17adec984affcdfb1618de89a5ddf4717cdf945cbadc059da67056ea73: Status 404 returned error can't find the container with id 66292d17adec984affcdfb1618de89a5ddf4717cdf945cbadc059da67056ea73 Jul 13 07:03:32.314115 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.312373 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 07:03:32.317585 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.317567 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.320460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.320439 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 07:03:32.320875 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.320822 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 07:03:32.320875 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.320842 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 07:03:32.320875 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.320856 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 07:03:32.321055 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.320830 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 07:03:32.321055 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.320937 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-f42rr\"" Jul 13 07:03:32.321055 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.320828 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 07:03:32.321055 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.320822 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 07:03:32.321198 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.321114 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 07:03:32.321259 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.321196 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 07:03:32.332513 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.332494 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 07:03:32.379588 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.379566 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:32.443679 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.443656 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:32.470120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470090 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470125 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470157 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-out\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470180 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-web-config\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470365 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470302 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470365 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470354 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470427 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470380 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470427 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470412 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470484 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470441 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470516 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470492 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470549 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470525 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470581 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470548 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6ht7\" (UniqueName: \"kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-kube-api-access-s6ht7\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.470581 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.470574 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571638 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571616 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571771 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571645 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571771 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571664 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6ht7\" (UniqueName: \"kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-kube-api-access-s6ht7\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571771 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571689 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571771 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571753 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571977 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571977 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571820 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-out\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571977 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571844 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-web-config\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571977 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571902 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571977 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571947 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.571977 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.571972 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.572299 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.572004 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.572299 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.572034 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.572299 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.572194 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.572971 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.572942 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.574126 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.574080 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.574535 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.574472 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-out\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.574639 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.574624 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.574928 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.574906 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.575038 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.575021 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.575535 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.575510 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.575667 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.575636 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.575667 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.575660 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.576378 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.576360 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-web-config\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.576636 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.576620 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.579432 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.579414 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6ht7\" (UniqueName: \"kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-kube-api-access-s6ht7\") pod \"alertmanager-main-0\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.626494 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.626444 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:03:32.693090 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.692998 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fpkv5" event={"ID":"13b90926-ab1e-4e33-a60e-45d5ba7d5abb","Type":"ContainerStarted","Data":"7e68fc711c2b7dd32d795df9f3a0920138fa25096a89f517d745619c5443727d"} Jul 13 07:03:32.694458 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.694427 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" event={"ID":"618475e3-496a-4bdc-bea8-8222dc14a179","Type":"ContainerStarted","Data":"66292d17adec984affcdfb1618de89a5ddf4717cdf945cbadc059da67056ea73"} Jul 13 07:03:32.696048 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.696023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lltp" event={"ID":"6b2a8f04-ea30-4461-a460-7fb3a891e063","Type":"ContainerStarted","Data":"449a613337a38c1fe92b02fe0463c58a904d047171d1873ca9000121bb2b64f9"} Jul 13 07:03:32.698325 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.698300 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" event={"ID":"f47ad689-c9b0-4f6f-9871-adec98f855fe","Type":"ContainerStarted","Data":"4e7dd9c95cc5ed01ac95f38b131d0faa86a7844899dfe19dc2a0608603e57b0a"} Jul 13 07:03:32.698403 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.698357 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" event={"ID":"f47ad689-c9b0-4f6f-9871-adec98f855fe","Type":"ContainerStarted","Data":"eaaa418615bff6aae16e2066a05a1a616059144e3fa304db1f6471bc18758918"} Jul 13 07:03:32.698403 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.698385 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" event={"ID":"f47ad689-c9b0-4f6f-9871-adec98f855fe","Type":"ContainerStarted","Data":"d1eca70b9e3a2c496e05994dada18715cba00f3990c3258758d4c70a5e795b2b"} Jul 13 07:03:32.710649 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.710515 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fpkv5" podStartSLOduration=1.640384633 podStartE2EDuration="10.710503265s" totalImagesPullingTime="9.070118632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:03:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:32.71037607 +0000 UTC m=+65.884879878" watchObservedRunningTime="2026-07-13 07:03:32.710503265 +0000 UTC m=+65.885007083" Jul 13 07:03:32.768503 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:32.768448 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 07:03:32.770507 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:32.770484 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f264c6_bba6_4e84_8350_8840b43f81a3.slice/crio-631383dd4467c125d31c13a66be011f6fffec9d2176021744a777be6c8153d1d WatchSource:0}: Error finding container 631383dd4467c125d31c13a66be011f6fffec9d2176021744a777be6c8153d1d: Status 404 returned error can't find the container with id 631383dd4467c125d31c13a66be011f6fffec9d2176021744a777be6c8153d1d Jul 13 07:03:33.179052 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.179016 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:33.179480 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.179229 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:33.182100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.182072 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 07:03:33.182100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.182089 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 07:03:33.192183 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.192160 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6d741e6-fdd4-442e-9940-2588e3ef9419-original-pull-secret\") pod \"global-pull-secret-syncer-b2qc9\" (UID: \"b6d741e6-fdd4-442e-9940-2588e3ef9419\") " pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:33.192327 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.192289 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a68e2b3a-fc60-4be7-9bf1-664317d72b94-metrics-certs\") pod \"network-metrics-daemon-jzrkj\" (UID: \"a68e2b3a-fc60-4be7-9bf1-664317d72b94\") " pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:33.213993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.213963 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-768d87b465-6tjct"] Jul 13 07:03:33.217270 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.217251 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.220424 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.220262 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 13 07:03:33.220424 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.220279 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 13 07:03:33.220424 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.220286 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 13 07:03:33.220424 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.220334 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 13 07:03:33.220424 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.220405 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8l228\"" Jul 13 07:03:33.220827 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.220625 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-mbvu6fnjl1kj\"" Jul 13 07:03:33.221022 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.221002 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 13 07:03:33.229076 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.229056 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-768d87b465-6tjct"] Jul 13 07:03:33.280092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.280070 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-metrics-client-ca\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.280233 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.280108 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.280233 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.280154 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-tls\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.280233 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.280190 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g7cj\" (UniqueName: \"kubernetes.io/projected/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-kube-api-access-9g7cj\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.280406 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.280264 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.280406 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.280300 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzn5\" (UniqueName: \"kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5\") pod \"network-check-target-f8xn4\" (UID: \"beb19682-944b-47da-b871-27e8f6dd449d\") " pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:33.280406 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.280336 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.280406 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.280361 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-grpc-tls\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.280406 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.280389 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.282821 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.282794 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 07:03:33.293144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.293127 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 07:03:33.304157 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.304132 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llzn5\" (UniqueName: \"kubernetes.io/projected/beb19682-944b-47da-b871-27e8f6dd449d-kube-api-access-llzn5\") pod \"network-check-target-f8xn4\" (UID: \"beb19682-944b-47da-b871-27e8f6dd449d\") " pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:33.341782 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.341709 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fh46b\"" Jul 13 07:03:33.349939 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.349911 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzrkj" Jul 13 07:03:33.351301 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.351101 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z8tdw\"" Jul 13 07:03:33.356238 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.356171 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2qc9" Jul 13 07:03:33.359080 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.359062 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:33.380990 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.380970 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-tls\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.381100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.381015 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g7cj\" (UniqueName: \"kubernetes.io/projected/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-kube-api-access-9g7cj\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.381100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.381069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.381278 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.381239 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.381414 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.381281 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-grpc-tls\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.381414 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.381313 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.381414 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.381349 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-metrics-client-ca\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.381414 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.381379 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.382226 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.382157 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-metrics-client-ca\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.384149 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.384126 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-grpc-tls\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.384283 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.384256 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.384497 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.384476 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-tls\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.384659 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.384619 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.385661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.385634 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.386782 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.386754 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.393768 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.393735 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g7cj\" (UniqueName: \"kubernetes.io/projected/84ddf01b-41e7-4c8d-8cc5-105691b08bd7-kube-api-access-9g7cj\") pod \"thanos-querier-768d87b465-6tjct\" (UID: \"84ddf01b-41e7-4c8d-8cc5-105691b08bd7\") " pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.489130 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.489100 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:33.527220 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.527151 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:33.588967 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.588929 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:33.647012 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.646938 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:33.703242 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.703186 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerStarted","Data":"631383dd4467c125d31c13a66be011f6fffec9d2176021744a777be6c8153d1d"} Jul 13 07:03:33.708767 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:33.708738 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:34.148031 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.148002 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:34.251812 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.251782 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:34.324226 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.324183 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jzrkj"] Jul 13 07:03:34.327324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.327258 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:34.409586 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.409550 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:34.411541 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:34.411506 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda68e2b3a_fc60_4be7_9bf1_664317d72b94.slice/crio-0b26ec01009b031dd4662268fd10690e34bb3b6a4bf844411823ca8e692fc210 WatchSource:0}: Error finding container 0b26ec01009b031dd4662268fd10690e34bb3b6a4bf844411823ca8e692fc210: Status 404 returned error can't find the container with id 0b26ec01009b031dd4662268fd10690e34bb3b6a4bf844411823ca8e692fc210 Jul 13 07:03:34.564985 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.564956 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b2qc9"] Jul 13 07:03:34.566436 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.566181 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f8xn4"] Jul 13 07:03:34.570967 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.570894 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-768d87b465-6tjct"] Jul 13 07:03:34.601099 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:34.601068 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d741e6_fdd4_442e_9940_2588e3ef9419.slice/crio-29ef1a6c4586b3efb748076651dc99a50923195d5f641903a58977768e6b540a WatchSource:0}: Error finding container 29ef1a6c4586b3efb748076651dc99a50923195d5f641903a58977768e6b540a: Status 404 returned error can't find the container with id 29ef1a6c4586b3efb748076651dc99a50923195d5f641903a58977768e6b540a Jul 13 07:03:34.608854 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:34.608833 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ddf01b_41e7_4c8d_8cc5_105691b08bd7.slice/crio-ee43099de81660e24a9885a342eaf3f8103e01dd39188e289c4370592bb27639 WatchSource:0}: Error finding container ee43099de81660e24a9885a342eaf3f8103e01dd39188e289c4370592bb27639: Status 404 returned error can't find the container with id ee43099de81660e24a9885a342eaf3f8103e01dd39188e289c4370592bb27639 Jul 13 07:03:34.708553 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.708519 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" event={"ID":"f47ad689-c9b0-4f6f-9871-adec98f855fe","Type":"ContainerStarted","Data":"8945b58182407ea3991173918352be90804955aec6772d6f8f7fda693b23fa98"} Jul 13 07:03:34.709762 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.709732 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" event={"ID":"84ddf01b-41e7-4c8d-8cc5-105691b08bd7","Type":"ContainerStarted","Data":"ee43099de81660e24a9885a342eaf3f8103e01dd39188e289c4370592bb27639"} Jul 13 07:03:34.711799 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.711774 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" event={"ID":"618475e3-496a-4bdc-bea8-8222dc14a179","Type":"ContainerStarted","Data":"8fe4e2ab486ddc76bcfec6804441ea76979881c7255e28ea28fa832941ef313e"} Jul 13 07:03:34.711899 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.711831 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" event={"ID":"618475e3-496a-4bdc-bea8-8222dc14a179","Type":"ContainerStarted","Data":"ae1aaad0cf05dfd605ff5b618e7abbcb214bbd3918ac25b17dc4fdf4c94274c1"} Jul 13 07:03:34.711899 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.711861 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" event={"ID":"618475e3-496a-4bdc-bea8-8222dc14a179","Type":"ContainerStarted","Data":"8112f72c3751b76a92095fb3a7a6f3c4aee5ddb586a13ac5f25ccd85406dbb3b"} Jul 13 07:03:34.713298 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.713266 2583 generic.go:356] "Generic (PLEG): container finished" podID="6b2a8f04-ea30-4461-a460-7fb3a891e063" containerID="e5163d8b2047cb22661e176f2ebed83b3f2449b19fdd8e8445ad650b4d183286" exitCode=0 Jul 13 07:03:34.713365 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.713333 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lltp" event={"ID":"6b2a8f04-ea30-4461-a460-7fb3a891e063","Type":"ContainerDied","Data":"e5163d8b2047cb22661e176f2ebed83b3f2449b19fdd8e8445ad650b4d183286"} Jul 13 07:03:34.714546 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.714519 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b2qc9" event={"ID":"b6d741e6-fdd4-442e-9940-2588e3ef9419","Type":"ContainerStarted","Data":"29ef1a6c4586b3efb748076651dc99a50923195d5f641903a58977768e6b540a"} Jul 13 07:03:34.715646 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.715623 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jzrkj" event={"ID":"a68e2b3a-fc60-4be7-9bf1-664317d72b94","Type":"ContainerStarted","Data":"0b26ec01009b031dd4662268fd10690e34bb3b6a4bf844411823ca8e692fc210"} Jul 13 07:03:34.716888 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.716868 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f8xn4" event={"ID":"beb19682-944b-47da-b871-27e8f6dd449d","Type":"ContainerStarted","Data":"36cd19afc9023da97e3de50f638991a1a3f2df87407f633306197bb491808a6a"} Jul 13 07:03:34.718236 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.718199 2583 generic.go:356] "Generic (PLEG): container finished" podID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerID="4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd" exitCode=0 Jul 13 07:03:34.718331 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.718246 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerDied","Data":"4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd"} Jul 13 07:03:34.725397 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.725242 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t8lb4" podStartSLOduration=1.961116963 podStartE2EDuration="3.725224375s" totalImagesPullingTime="1.764107412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:34.724941379 +0000 UTC m=+67.899445186" watchObservedRunningTime="2026-07-13 07:03:34.725224375 +0000 UTC m=+67.899728183" Jul 13 07:03:34.729338 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.729315 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:34.763169 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.763126 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-6cvmw" podStartSLOduration=1.910030807 podStartE2EDuration="3.763109545s" totalImagesPullingTime="1.853078738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:34.762381674 +0000 UTC m=+67.936885481" watchObservedRunningTime="2026-07-13 07:03:34.763109545 +0000 UTC m=+67.937613355" Jul 13 07:03:34.811266 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:34.808444 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:35.479034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.478994 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:35.580199 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.580167 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:35.652045 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.652008 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5d66dd77b8-7svz5"] Jul 13 07:03:35.669020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.668961 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d66dd77b8-7svz5"] Jul 13 07:03:35.669187 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.669145 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.673890 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.673830 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 13 07:03:35.674046 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.674011 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 13 07:03:35.674365 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.674265 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-42abla841t35k\"" Jul 13 07:03:35.674365 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.674323 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 13 07:03:35.674526 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.674505 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 13 07:03:35.674894 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.674684 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2r5zg\"" Jul 13 07:03:35.726493 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.725851 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lltp" event={"ID":"6b2a8f04-ea30-4461-a460-7fb3a891e063","Type":"ContainerStarted","Data":"14c4779ac1f9965c06c626d8be872528795b1e8594e8cff60831d4ff9b617854"} Jul 13 07:03:35.726493 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.725905 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lltp" event={"ID":"6b2a8f04-ea30-4461-a460-7fb3a891e063","Type":"ContainerStarted","Data":"92fca7615c21a78ba9e6223f143967f477c5c050170f939f8ae05889438b194a"} Jul 13 07:03:35.746968 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.745001 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8lltp" podStartSLOduration=2.646126903 podStartE2EDuration="4.744982432s" totalImagesPullingTime="1.980988235s" totalInitContainerRuntime="117.867294ms" isStatefulPod=true podCreationTimestamp="2026-07-13 07:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:35.744362817 +0000 UTC m=+68.918866646" watchObservedRunningTime="2026-07-13 07:03:35.744982432 +0000 UTC m=+68.919486238" Jul 13 07:03:35.803254 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.803140 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8694a18f-2def-4299-b570-b0c4df0c21cc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.803254 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.803233 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8694a18f-2def-4299-b570-b0c4df0c21cc-secret-metrics-server-tls\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.803458 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.803285 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8694a18f-2def-4299-b570-b0c4df0c21cc-audit-log\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.803458 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.803431 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8694a18f-2def-4299-b570-b0c4df0c21cc-client-ca-bundle\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.803563 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.803476 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftqrq\" (UniqueName: \"kubernetes.io/projected/8694a18f-2def-4299-b570-b0c4df0c21cc-kube-api-access-ftqrq\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.803618 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.803587 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8694a18f-2def-4299-b570-b0c4df0c21cc-secret-metrics-server-client-certs\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.803618 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.803613 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8694a18f-2def-4299-b570-b0c4df0c21cc-metrics-server-audit-profiles\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.907117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.904188 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8694a18f-2def-4299-b570-b0c4df0c21cc-client-ca-bundle\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.907117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.904255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftqrq\" (UniqueName: \"kubernetes.io/projected/8694a18f-2def-4299-b570-b0c4df0c21cc-kube-api-access-ftqrq\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.907117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.904300 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8694a18f-2def-4299-b570-b0c4df0c21cc-secret-metrics-server-client-certs\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.907117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.904325 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8694a18f-2def-4299-b570-b0c4df0c21cc-metrics-server-audit-profiles\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.907117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.904363 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8694a18f-2def-4299-b570-b0c4df0c21cc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.907117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.904401 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8694a18f-2def-4299-b570-b0c4df0c21cc-secret-metrics-server-tls\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.907117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.904431 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8694a18f-2def-4299-b570-b0c4df0c21cc-audit-log\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.907117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.904804 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8694a18f-2def-4299-b570-b0c4df0c21cc-audit-log\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.907117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.906063 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8694a18f-2def-4299-b570-b0c4df0c21cc-metrics-server-audit-profiles\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.907117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.906684 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8694a18f-2def-4299-b570-b0c4df0c21cc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.920256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.910973 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8694a18f-2def-4299-b570-b0c4df0c21cc-client-ca-bundle\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.920256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.917737 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8694a18f-2def-4299-b570-b0c4df0c21cc-secret-metrics-server-client-certs\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.920256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.920151 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftqrq\" (UniqueName: \"kubernetes.io/projected/8694a18f-2def-4299-b570-b0c4df0c21cc-kube-api-access-ftqrq\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.921034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.920994 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8694a18f-2def-4299-b570-b0c4df0c21cc-secret-metrics-server-tls\") pod \"metrics-server-5d66dd77b8-7svz5\" (UID: \"8694a18f-2def-4299-b570-b0c4df0c21cc\") " pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.982073 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.981764 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:35.997238 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:35.997113 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:36.015672 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.014455 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp"] Jul 13 07:03:36.034467 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.034387 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp"] Jul 13 07:03:36.034762 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.034701 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" Jul 13 07:03:36.038701 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.038669 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 13 07:03:36.038970 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.038953 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-94r4q\"" Jul 13 07:03:36.056316 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.056287 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:36.108043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.107650 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/800b08e1-c950-4dcf-a73e-3eebdfc3f5d1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mmsrp\" (UID: \"800b08e1-c950-4dcf-a73e-3eebdfc3f5d1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" Jul 13 07:03:36.108043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.107750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bps8m\" (UniqueName: \"kubernetes.io/projected/800b08e1-c950-4dcf-a73e-3eebdfc3f5d1-kube-api-access-bps8m\") pod \"monitoring-plugin-5f68df7f58-mmsrp\" (UID: \"800b08e1-c950-4dcf-a73e-3eebdfc3f5d1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" Jul 13 07:03:36.190001 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.189928 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d66dd77b8-7svz5"] Jul 13 07:03:36.193227 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:36.193178 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8694a18f_2def_4299_b570_b0c4df0c21cc.slice/crio-ae74031d5ce0caf4cbc4a8f52a2dba638d6c1ba578409dd7972799de06f99988 WatchSource:0}: Error finding container ae74031d5ce0caf4cbc4a8f52a2dba638d6c1ba578409dd7972799de06f99988: Status 404 returned error can't find the container with id ae74031d5ce0caf4cbc4a8f52a2dba638d6c1ba578409dd7972799de06f99988 Jul 13 07:03:36.195898 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.195846 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:36.209516 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.208828 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bps8m\" (UniqueName: \"kubernetes.io/projected/800b08e1-c950-4dcf-a73e-3eebdfc3f5d1-kube-api-access-bps8m\") pod \"monitoring-plugin-5f68df7f58-mmsrp\" (UID: \"800b08e1-c950-4dcf-a73e-3eebdfc3f5d1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" Jul 13 07:03:36.209516 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.208898 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/800b08e1-c950-4dcf-a73e-3eebdfc3f5d1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mmsrp\" (UID: \"800b08e1-c950-4dcf-a73e-3eebdfc3f5d1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" Jul 13 07:03:36.213008 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.212927 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/800b08e1-c950-4dcf-a73e-3eebdfc3f5d1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mmsrp\" (UID: \"800b08e1-c950-4dcf-a73e-3eebdfc3f5d1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" Jul 13 07:03:36.226689 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.226639 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bps8m\" (UniqueName: \"kubernetes.io/projected/800b08e1-c950-4dcf-a73e-3eebdfc3f5d1-kube-api-access-bps8m\") pod \"monitoring-plugin-5f68df7f58-mmsrp\" (UID: \"800b08e1-c950-4dcf-a73e-3eebdfc3f5d1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" Jul 13 07:03:36.354731 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.354657 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" Jul 13 07:03:36.449450 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.449379 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5fd557576d-xnvk5"] Jul 13 07:03:36.468336 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.466241 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.468336 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.467406 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5fd557576d-xnvk5"] Jul 13 07:03:36.470493 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.469526 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 07:03:36.470493 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.469764 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 07:03:36.470493 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.469990 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 07:03:36.470493 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.470173 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lt9zb\"" Jul 13 07:03:36.470493 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.470388 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 07:03:36.492066 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.492038 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 07:03:36.501385 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.501353 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 07:03:36.530396 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.530376 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp"] Jul 13 07:03:36.614237 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.614146 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fc1a8f5-38b0-4326-8722-451f9258c1f0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.614237 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.614195 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-telemeter-client-tls\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.614425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.614242 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.614425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.614293 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-federate-client-tls\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.614425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.614320 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-secret-telemeter-client\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.614425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.614345 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fc1a8f5-38b0-4326-8722-451f9258c1f0-metrics-client-ca\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.614425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.614386 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9llvl\" (UniqueName: \"kubernetes.io/projected/2fc1a8f5-38b0-4326-8722-451f9258c1f0-kube-api-access-9llvl\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.614425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.614424 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fc1a8f5-38b0-4326-8722-451f9258c1f0-serving-certs-ca-bundle\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.715830 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.715796 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-federate-client-tls\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.715967 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.715847 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-secret-telemeter-client\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.715967 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.715877 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fc1a8f5-38b0-4326-8722-451f9258c1f0-metrics-client-ca\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.715967 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.715920 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9llvl\" (UniqueName: \"kubernetes.io/projected/2fc1a8f5-38b0-4326-8722-451f9258c1f0-kube-api-access-9llvl\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.715967 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.715953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fc1a8f5-38b0-4326-8722-451f9258c1f0-serving-certs-ca-bundle\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.716256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.716023 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fc1a8f5-38b0-4326-8722-451f9258c1f0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.716256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.716053 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-telemeter-client-tls\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.716256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.716078 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.718958 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.717993 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fc1a8f5-38b0-4326-8722-451f9258c1f0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.718958 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.718678 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fc1a8f5-38b0-4326-8722-451f9258c1f0-serving-certs-ca-bundle\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.718958 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.718905 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fc1a8f5-38b0-4326-8722-451f9258c1f0-metrics-client-ca\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.720132 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.720072 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-federate-client-tls\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.720776 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.720736 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-secret-telemeter-client\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.721609 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.721568 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-telemeter-client-tls\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.722403 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.722364 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fc1a8f5-38b0-4326-8722-451f9258c1f0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.729302 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.729259 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9llvl\" (UniqueName: \"kubernetes.io/projected/2fc1a8f5-38b0-4326-8722-451f9258c1f0-kube-api-access-9llvl\") pod \"telemeter-client-5fd557576d-xnvk5\" (UID: \"2fc1a8f5-38b0-4326-8722-451f9258c1f0\") " pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.733561 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.733528 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jzrkj" event={"ID":"a68e2b3a-fc60-4be7-9bf1-664317d72b94","Type":"ContainerStarted","Data":"2eced44f87f0a8cbb98a3beea4909cf08fc8217c4f3f039ef3e93cb5de56b29a"} Jul 13 07:03:36.733648 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.733595 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jzrkj" event={"ID":"a68e2b3a-fc60-4be7-9bf1-664317d72b94","Type":"ContainerStarted","Data":"57ee552efda09fd8aad1d70f7fed3944a8833606801bc1132f3ce2ad341e4b4a"} Jul 13 07:03:36.737167 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.737125 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" event={"ID":"8694a18f-2def-4299-b570-b0c4df0c21cc","Type":"ContainerStarted","Data":"ae74031d5ce0caf4cbc4a8f52a2dba638d6c1ba578409dd7972799de06f99988"} Jul 13 07:03:36.752450 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.752400 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jzrkj" podStartSLOduration=67.276744654 podStartE2EDuration="1m8.752382368s" totalImagesPullingTime="1.475637714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:36.751124309 +0000 UTC m=+69.925628118" watchObservedRunningTime="2026-07-13 07:03:36.752382368 +0000 UTC m=+69.926886176" Jul 13 07:03:36.805119 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.805097 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" Jul 13 07:03:36.846807 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.846661 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:36.913165 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:36.913091 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:37.154810 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:37.154776 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800b08e1_c950_4dcf_a73e_3eebdfc3f5d1.slice/crio-7d1579383a01fd2dd7dfd6dcfef2d9af881fe7a93ef146b838ba1e9d2d101d1d WatchSource:0}: Error finding container 7d1579383a01fd2dd7dfd6dcfef2d9af881fe7a93ef146b838ba1e9d2d101d1d: Status 404 returned error can't find the container with id 7d1579383a01fd2dd7dfd6dcfef2d9af881fe7a93ef146b838ba1e9d2d101d1d Jul 13 07:03:37.353148 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.353107 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65ff898685-bnprj"] Jul 13 07:03:37.355869 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.355761 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.359092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.358816 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cxpws\"" Jul 13 07:03:37.359503 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.359484 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 07:03:37.360412 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.360380 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 07:03:37.361034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.360965 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 07:03:37.361681 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.361665 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 07:03:37.362076 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.361847 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 07:03:37.362076 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.361918 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 07:03:37.362076 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.361778 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 07:03:37.366824 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.366800 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65ff898685-bnprj"] Jul 13 07:03:37.368847 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.368825 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 07:03:37.470232 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.469780 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:37.475129 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.475101 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 07:03:37.478788 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.478765 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.484581 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.483992 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 13 07:03:37.484581 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.484122 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 13 07:03:37.484581 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.484233 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 13 07:03:37.484581 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.484414 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 13 07:03:37.484581 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.484567 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 13 07:03:37.486036 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.485324 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 13 07:03:37.486036 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.485608 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 13 07:03:37.486036 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.485724 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 13 07:03:37.494400 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.487876 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 13 07:03:37.494400 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.488148 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cm7rf32q20esh\"" Jul 13 07:03:37.494400 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.488439 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 13 07:03:37.494400 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.488673 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 13 07:03:37.494400 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.492379 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kzqvw\"" Jul 13 07:03:37.494400 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.492706 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 13 07:03:37.498836 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.494714 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 13 07:03:37.500982 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.500770 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 13 07:03:37.504319 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.504285 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 07:03:37.528587 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.528555 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsm76\" (UniqueName: \"kubernetes.io/projected/c761caa1-77da-4751-87a9-3ea83266e576-kube-api-access-wsm76\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.528721 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.528601 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-serving-cert\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.528721 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.528670 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-oauth-serving-cert\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.528721 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.528704 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-trusted-ca-bundle\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.528894 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.528806 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-oauth-config\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.528894 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.528859 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-console-config\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.528989 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.528910 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-service-ca\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.630185 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630105 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630185 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630165 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630194 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630243 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-oauth-config\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.630411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630269 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-console-config\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.630411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630293 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630321 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630348 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630386 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630414 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630438 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630465 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-service-ca\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630489 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-config\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630510 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630539 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsm76\" (UniqueName: \"kubernetes.io/projected/c761caa1-77da-4751-87a9-3ea83266e576-kube-api-access-wsm76\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630565 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-serving-cert\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630612 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-config-out\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630635 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630662 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-web-config\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630691 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630722 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-oauth-serving-cert\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.630753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630744 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s2f8\" (UniqueName: \"kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-kube-api-access-2s2f8\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.631440 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630782 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-trusted-ca-bundle\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.631440 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630805 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.631440 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630829 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.631440 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630859 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.631440 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.630883 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.632661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.632368 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-console-config\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.632661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.632460 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-trusted-ca-bundle\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.632661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.632599 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-service-ca\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.632867 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.632744 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-oauth-serving-cert\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.635737 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.634684 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-oauth-config\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.635737 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.634684 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-serving-cert\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.638859 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.638819 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:37.643483 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.643447 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsm76\" (UniqueName: \"kubernetes.io/projected/c761caa1-77da-4751-87a9-3ea83266e576-kube-api-access-wsm76\") pod \"console-65ff898685-bnprj\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.678734 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.678706 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732364 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732410 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732446 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732477 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732504 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732534 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-config\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732613 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-config-out\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732639 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732664 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-web-config\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732728 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s2f8\" (UniqueName: \"kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-kube-api-access-2s2f8\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732765 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732788 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732817 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.732839 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.734223 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.733792 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.734396 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.734746 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.735119 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.735280 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.735870 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.737173 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.737252 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.737307 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.737337 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.737366 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.737994 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.738692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.738266 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-config\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.741738 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.741696 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.744225 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.744086 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:37.745949 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.745907 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s2f8\" (UniqueName: \"kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-kube-api-access-2s2f8\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.746546 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.746498 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.747383 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.747287 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.750588 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.750534 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.751365 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.750899 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-config-out\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.751365 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.751138 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.751365 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.751315 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.751642 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.751549 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.751642 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.751551 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-web-config\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.754558 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.752745 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:37.754558 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.754490 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" event={"ID":"800b08e1-c950-4dcf-a73e-3eebdfc3f5d1","Type":"ContainerStarted","Data":"7d1579383a01fd2dd7dfd6dcfef2d9af881fe7a93ef146b838ba1e9d2d101d1d"} Jul 13 07:03:37.804817 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:37.804539 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:38.377644 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:38.377557 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:38.680317 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:38.680245 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9ttvg" Jul 13 07:03:38.915122 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:38.915074 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:39.016494 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:39.016459 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:40.511338 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:40.511312 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 07:03:41.096462 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:41.096430 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d23102_346e_4d51_aa07_bd4632ebe863.slice/crio-b57488f68498e5a04b4d16634659471d01a870f869f25bf408f4aa678b1f602b WatchSource:0}: Error finding container b57488f68498e5a04b4d16634659471d01a870f869f25bf408f4aa678b1f602b: Status 404 returned error can't find the container with id b57488f68498e5a04b4d16634659471d01a870f869f25bf408f4aa678b1f602b Jul 13 07:03:41.328497 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.327819 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5fd557576d-xnvk5"] Jul 13 07:03:41.333717 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:41.333689 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc1a8f5_38b0_4326_8722_451f9258c1f0.slice/crio-a9fdacb610850cda548d04b455334dc74279eae46825b3f8c209774d392bffe9 WatchSource:0}: Error finding container a9fdacb610850cda548d04b455334dc74279eae46825b3f8c209774d392bffe9: Status 404 returned error can't find the container with id a9fdacb610850cda548d04b455334dc74279eae46825b3f8c209774d392bffe9 Jul 13 07:03:41.344731 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.344661 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65ff898685-bnprj"] Jul 13 07:03:41.399850 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:03:41.399821 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc761caa1_77da_4751_87a9_3ea83266e576.slice/crio-c42793dceaf47e0c72cdacd03295d2154a1e103d9c05f3072e4f41ce00d432b7 WatchSource:0}: Error finding container c42793dceaf47e0c72cdacd03295d2154a1e103d9c05f3072e4f41ce00d432b7: Status 404 returned error can't find the container with id c42793dceaf47e0c72cdacd03295d2154a1e103d9c05f3072e4f41ce00d432b7 Jul 13 07:03:41.435376 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.435349 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:41.535280 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.535245 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:41.719763 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.719735 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:41.771658 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.771627 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f8xn4" event={"ID":"beb19682-944b-47da-b871-27e8f6dd449d","Type":"ContainerStarted","Data":"44c6b76d2b1a8bfb41c0bf6b9c89ada477449b3e6b34b4ab11c7ef0219df0da3"} Jul 13 07:03:41.771773 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.771739 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:03:41.773121 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.773098 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" event={"ID":"8694a18f-2def-4299-b570-b0c4df0c21cc","Type":"ContainerStarted","Data":"ff2e3a227cceea28cc15b30094b5dd45f8e8db836cf4c7b9fadecc36c8dc2a6a"} Jul 13 07:03:41.776345 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.776324 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerStarted","Data":"82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba"} Jul 13 07:03:41.776444 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.776375 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerStarted","Data":"d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a"} Jul 13 07:03:41.776444 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.776402 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerStarted","Data":"a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27"} Jul 13 07:03:41.776444 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.776435 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerStarted","Data":"2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0"} Jul 13 07:03:41.776650 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.776473 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerStarted","Data":"ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465"} Jul 13 07:03:41.777803 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.777779 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" event={"ID":"800b08e1-c950-4dcf-a73e-3eebdfc3f5d1","Type":"ContainerStarted","Data":"2f9264b3c2eae29d4611fa69c9e1da13db739835dadb7971f1506105365f6753"} Jul 13 07:03:41.777965 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.777945 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" Jul 13 07:03:41.779164 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.779141 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65ff898685-bnprj" event={"ID":"c761caa1-77da-4751-87a9-3ea83266e576","Type":"ContainerStarted","Data":"c42793dceaf47e0c72cdacd03295d2154a1e103d9c05f3072e4f41ce00d432b7"} Jul 13 07:03:41.781245 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.781224 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" event={"ID":"84ddf01b-41e7-4c8d-8cc5-105691b08bd7","Type":"ContainerStarted","Data":"31669bcda7e3254668e4b017f443c729e72c5e7d3b83783fa62db9a98e506f6c"} Jul 13 07:03:41.781337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.781288 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" event={"ID":"84ddf01b-41e7-4c8d-8cc5-105691b08bd7","Type":"ContainerStarted","Data":"afb532940946587dded528269a17cc9970aaf2ee04d3f6c8a5e6574a77cb817d"} Jul 13 07:03:41.781337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.781327 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" event={"ID":"84ddf01b-41e7-4c8d-8cc5-105691b08bd7","Type":"ContainerStarted","Data":"43cf8e17a879578d7e8d96bc1e25efe94a927a923a410c091a4bbf9d0327946f"} Jul 13 07:03:41.782439 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.782414 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" event={"ID":"2fc1a8f5-38b0-4326-8722-451f9258c1f0","Type":"ContainerStarted","Data":"a9fdacb610850cda548d04b455334dc74279eae46825b3f8c209774d392bffe9"} Jul 13 07:03:41.783788 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.783770 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" Jul 13 07:03:41.784391 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.784372 2583 generic.go:356] "Generic (PLEG): container finished" podID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerID="99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca" exitCode=0 Jul 13 07:03:41.784478 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.784412 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerDied","Data":"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca"} Jul 13 07:03:41.784478 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.784462 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerStarted","Data":"b57488f68498e5a04b4d16634659471d01a870f869f25bf408f4aa678b1f602b"} Jul 13 07:03:41.785917 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.785897 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b2qc9" event={"ID":"b6d741e6-fdd4-442e-9940-2588e3ef9419","Type":"ContainerStarted","Data":"31bcf190bcbeb956e1badce2937aea50f22605628fb008ef1d0f4dfb139ebfb0"} Jul 13 07:03:41.791398 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.791363 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f8xn4" podStartSLOduration=67.254791163 podStartE2EDuration="1m13.791350111s" totalImagesPullingTime="6.536558948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:41.79063222 +0000 UTC m=+74.965136027" watchObservedRunningTime="2026-07-13 07:03:41.791350111 +0000 UTC m=+74.965853917" Jul 13 07:03:41.801319 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.801300 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:41.809426 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.809393 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" podStartSLOduration=1.872645898 podStartE2EDuration="6.809381154s" totalImagesPullingTime="4.936735256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:41.808057483 +0000 UTC m=+74.982561291" watchObservedRunningTime="2026-07-13 07:03:41.809381154 +0000 UTC m=+74.983884960" Jul 13 07:03:41.860139 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.860042 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mmsrp" podStartSLOduration=2.879453529 podStartE2EDuration="6.860029012s" totalImagesPullingTime="3.980575483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:41.858546807 +0000 UTC m=+75.033050625" watchObservedRunningTime="2026-07-13 07:03:41.860029012 +0000 UTC m=+75.034532797" Jul 13 07:03:41.880408 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:41.880360 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b2qc9" podStartSLOduration=67.350100824 podStartE2EDuration="1m13.880339571s" totalImagesPullingTime="6.530238747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:41.879092076 +0000 UTC m=+75.053595884" watchObservedRunningTime="2026-07-13 07:03:41.880339571 +0000 UTC m=+75.054843379" Jul 13 07:03:42.796770 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:42.796525 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerStarted","Data":"462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593"} Jul 13 07:03:42.803389 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:42.801784 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" event={"ID":"84ddf01b-41e7-4c8d-8cc5-105691b08bd7","Type":"ContainerStarted","Data":"3329e4aac65ae7cf309dd17be8cebb38d7aa24efe97b47f6a771168904360aa9"} Jul 13 07:03:42.803389 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:42.801848 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" event={"ID":"84ddf01b-41e7-4c8d-8cc5-105691b08bd7","Type":"ContainerStarted","Data":"c8ec42ea4af9585161b2826c153b4b53d4d2c19a9ee8fe2fc641001a6a391815"} Jul 13 07:03:42.803389 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:42.801890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" event={"ID":"84ddf01b-41e7-4c8d-8cc5-105691b08bd7","Type":"ContainerStarted","Data":"b648fc3219842d3eac05f7672eafb18079cbbb5ba35ca03f319682a9133ff10e"} Jul 13 07:03:42.803389 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:42.802476 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:42.827002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:42.826950 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.081358927 podStartE2EDuration="10.826930317s" totalImagesPullingTime="8.724792343s" totalInitContainerRuntime="20.779047ms" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:42.825118453 +0000 UTC m=+75.999622262" watchObservedRunningTime="2026-07-13 07:03:42.826930317 +0000 UTC m=+76.001434127" Jul 13 07:03:42.852054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:42.850958 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" podStartSLOduration=3.240531993 podStartE2EDuration="9.850939935s" totalImagesPullingTime="6.610407942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:42.849288666 +0000 UTC m=+76.023792482" watchObservedRunningTime="2026-07-13 07:03:42.850939935 +0000 UTC m=+76.025443745" Jul 13 07:03:45.218455 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:45.218417 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:45.324584 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:45.324547 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:45.826475 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:45.826442 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65ff898685-bnprj" event={"ID":"c761caa1-77da-4751-87a9-3ea83266e576","Type":"ContainerStarted","Data":"2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5"} Jul 13 07:03:45.828799 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:45.828737 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" event={"ID":"2fc1a8f5-38b0-4326-8722-451f9258c1f0","Type":"ContainerStarted","Data":"41a2e1c90e659b8bfa49a3ba1c4927892dcae7e2510675367b64732c0a592a12"} Jul 13 07:03:45.831562 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:45.831544 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerStarted","Data":"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c"} Jul 13 07:03:45.856048 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:45.855764 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65ff898685-bnprj" podStartSLOduration=4.535640482 podStartE2EDuration="8.855749008s" totalImagesPullingTime="4.320108526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:45.853584612 +0000 UTC m=+79.028088441" watchObservedRunningTime="2026-07-13 07:03:45.855749008 +0000 UTC m=+79.030252816" Jul 13 07:03:46.837969 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:46.837925 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" event={"ID":"2fc1a8f5-38b0-4326-8722-451f9258c1f0","Type":"ContainerStarted","Data":"3275c79c0522a8e1f3da76b3525ec1111718f57b487a28eb776446457bf1d03b"} Jul 13 07:03:46.838391 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:46.838006 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" event={"ID":"2fc1a8f5-38b0-4326-8722-451f9258c1f0","Type":"ContainerStarted","Data":"58ad86aac58b75b85b353765724c19b25f70d2e48c9206c72170bf66f7f6f394"} Jul 13 07:03:46.841322 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:46.841299 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerStarted","Data":"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a"} Jul 13 07:03:46.841428 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:46.841349 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerStarted","Data":"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897"} Jul 13 07:03:46.841428 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:46.841374 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerStarted","Data":"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e"} Jul 13 07:03:46.841428 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:46.841403 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerStarted","Data":"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a"} Jul 13 07:03:46.841533 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:46.841431 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerStarted","Data":"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822"} Jul 13 07:03:46.864464 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:46.864419 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5fd557576d-xnvk5" podStartSLOduration=6.485918408 podStartE2EDuration="10.864405709s" totalImagesPullingTime="4.378487301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:46.86239264 +0000 UTC m=+80.036896447" watchObservedRunningTime="2026-07-13 07:03:46.864405709 +0000 UTC m=+80.038909516" Jul 13 07:03:46.898006 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:46.897965 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.915827812 podStartE2EDuration="9.897951839s" totalImagesPullingTime="3.946158273s" totalInitContainerRuntime="35.965754ms" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:46.895775183 +0000 UTC m=+80.070279030" watchObservedRunningTime="2026-07-13 07:03:46.897951839 +0000 UTC m=+80.072455645" Jul 13 07:03:47.399306 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:47.399274 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:47.501338 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:47.501311 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:47.679788 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:47.679710 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:47.679788 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:47.679767 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:47.685254 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:47.685225 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:47.805179 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:47.805146 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:03:47.851004 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:47.850981 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:03:49.818635 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:49.818526 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-768d87b465-6tjct" Jul 13 07:03:50.809924 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:50.809892 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:50.910158 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:50.910132 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:55.982286 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:55.982254 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:55.982642 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:55.982307 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:03:56.438536 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:56.438449 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:56.494424 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:56.494391 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:59.025643 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:59.025610 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65ff898685-bnprj"] Jul 13 07:03:59.128240 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:59.128197 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:59.185856 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:59.185811 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:03:59.285786 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:03:59.285714 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:07.084551 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:07.084515 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:08.878919 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:08.878886 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:08.931422 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:08.931396 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:11.445816 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:11.445782 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:11.506063 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:11.506034 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:12.804977 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:12.804940 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f8xn4" Jul 13 07:04:15.988417 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:15.988384 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:04:15.993067 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:15.993043 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5d66dd77b8-7svz5" Jul 13 07:04:16.413432 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:16.413345 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:24.046848 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.046777 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65ff898685-bnprj" podUID="c761caa1-77da-4751-87a9-3ea83266e576" containerName="console" containerID="cri-o://2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5" gracePeriod=15 Jul 13 07:04:24.283505 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.283476 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:04:24.420985 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.420927 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-oauth-config\") pod \"c761caa1-77da-4751-87a9-3ea83266e576\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " Jul 13 07:04:24.420985 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.420959 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-console-config\") pod \"c761caa1-77da-4751-87a9-3ea83266e576\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " Jul 13 07:04:24.421162 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.420990 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-oauth-serving-cert\") pod \"c761caa1-77da-4751-87a9-3ea83266e576\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " Jul 13 07:04:24.421162 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.421015 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-service-ca\") pod \"c761caa1-77da-4751-87a9-3ea83266e576\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " Jul 13 07:04:24.421162 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.421069 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsm76\" (UniqueName: \"kubernetes.io/projected/c761caa1-77da-4751-87a9-3ea83266e576-kube-api-access-wsm76\") pod \"c761caa1-77da-4751-87a9-3ea83266e576\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " Jul 13 07:04:24.421162 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.421095 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-trusted-ca-bundle\") pod \"c761caa1-77da-4751-87a9-3ea83266e576\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " Jul 13 07:04:24.421162 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.421131 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-serving-cert\") pod \"c761caa1-77da-4751-87a9-3ea83266e576\" (UID: \"c761caa1-77da-4751-87a9-3ea83266e576\") " Jul 13 07:04:24.421461 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.421424 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-console-config" (OuterVolumeSpecName: "console-config") pod "c761caa1-77da-4751-87a9-3ea83266e576" (UID: "c761caa1-77da-4751-87a9-3ea83266e576"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:24.421517 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.421451 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c761caa1-77da-4751-87a9-3ea83266e576" (UID: "c761caa1-77da-4751-87a9-3ea83266e576"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:24.421517 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.421485 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-service-ca" (OuterVolumeSpecName: "service-ca") pod "c761caa1-77da-4751-87a9-3ea83266e576" (UID: "c761caa1-77da-4751-87a9-3ea83266e576"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:24.421587 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.421516 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c761caa1-77da-4751-87a9-3ea83266e576" (UID: "c761caa1-77da-4751-87a9-3ea83266e576"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:24.423395 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.423367 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c761caa1-77da-4751-87a9-3ea83266e576-kube-api-access-wsm76" (OuterVolumeSpecName: "kube-api-access-wsm76") pod "c761caa1-77da-4751-87a9-3ea83266e576" (UID: "c761caa1-77da-4751-87a9-3ea83266e576"). InnerVolumeSpecName "kube-api-access-wsm76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:04:24.423395 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.423382 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c761caa1-77da-4751-87a9-3ea83266e576" (UID: "c761caa1-77da-4751-87a9-3ea83266e576"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:24.423514 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.423475 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c761caa1-77da-4751-87a9-3ea83266e576" (UID: "c761caa1-77da-4751-87a9-3ea83266e576"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:24.522656 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.522636 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsm76\" (UniqueName: \"kubernetes.io/projected/c761caa1-77da-4751-87a9-3ea83266e576-kube-api-access-wsm76\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:24.522656 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.522656 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-trusted-ca-bundle\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:24.522769 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.522665 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-serving-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:24.522769 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.522673 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c761caa1-77da-4751-87a9-3ea83266e576-console-oauth-config\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:24.522769 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.522682 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-console-config\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:24.522769 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.522689 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-oauth-serving-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:24.522769 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:24.522697 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c761caa1-77da-4751-87a9-3ea83266e576-service-ca\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:25.006222 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:25.006174 2583 generic.go:356] "Generic (PLEG): container finished" podID="c761caa1-77da-4751-87a9-3ea83266e576" containerID="2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5" exitCode=0 Jul 13 07:04:25.006411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:25.006239 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65ff898685-bnprj" Jul 13 07:04:25.006411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:25.006258 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65ff898685-bnprj" event={"ID":"c761caa1-77da-4751-87a9-3ea83266e576","Type":"ContainerDied","Data":"2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5"} Jul 13 07:04:25.006411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:25.006308 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65ff898685-bnprj" event={"ID":"c761caa1-77da-4751-87a9-3ea83266e576","Type":"ContainerDied","Data":"c42793dceaf47e0c72cdacd03295d2154a1e103d9c05f3072e4f41ce00d432b7"} Jul 13 07:04:25.006411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:25.006328 2583 scope.go:117] "RemoveContainer" containerID="2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5" Jul 13 07:04:25.016255 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:25.016237 2583 scope.go:117] "RemoveContainer" containerID="2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5" Jul 13 07:04:25.016507 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:25.016486 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5\": container with ID starting with 2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5 not found: ID does not exist" containerID="2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5" Jul 13 07:04:25.016562 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:25.016518 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5"} err="failed to get container status \"2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5\": rpc error: code = NotFound desc = could not find container \"2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5\": container with ID starting with 2a3751c97113e673d710d51b4034258f991d7bdbb5b701b86806369692f02af5 not found: ID does not exist" Jul 13 07:04:25.027156 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:25.027135 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65ff898685-bnprj"] Jul 13 07:04:25.030445 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:25.030426 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65ff898685-bnprj"] Jul 13 07:04:25.427075 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:25.427002 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c761caa1-77da-4751-87a9-3ea83266e576" path="/var/lib/kubelet/pods/c761caa1-77da-4751-87a9-3ea83266e576/volumes" Jul 13 07:04:37.805685 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:37.805652 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:37.823928 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:37.823904 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:38.069938 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:38.069874 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:47.308261 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:47.308219 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:47.408525 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:47.408498 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:50.027118 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:50.027083 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:50.079242 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:50.079196 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:50.176431 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:50.176408 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:50.231516 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:50.231495 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:50.770172 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:50.770148 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:50.827140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:50.827117 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:51.564463 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:51.564430 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 07:04:51.564885 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:51.564767 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="alertmanager" containerID="cri-o://ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465" gracePeriod=120 Jul 13 07:04:51.564885 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:51.564819 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy-metric" containerID="cri-o://82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba" gracePeriod=120 Jul 13 07:04:51.564885 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:51.564861 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="config-reloader" containerID="cri-o://2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0" gracePeriod=120 Jul 13 07:04:51.565049 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:51.564859 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy-web" containerID="cri-o://a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27" gracePeriod=120 Jul 13 07:04:51.565049 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:51.564909 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy" containerID="cri-o://d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a" gracePeriod=120 Jul 13 07:04:51.565049 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:51.564934 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="prom-label-proxy" containerID="cri-o://462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593" gracePeriod=120 Jul 13 07:04:52.109282 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.109257 2583 generic.go:356] "Generic (PLEG): container finished" podID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerID="462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593" exitCode=0 Jul 13 07:04:52.109282 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.109280 2583 generic.go:356] "Generic (PLEG): container finished" podID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerID="82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba" exitCode=0 Jul 13 07:04:52.109282 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.109286 2583 generic.go:356] "Generic (PLEG): container finished" podID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerID="d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a" exitCode=0 Jul 13 07:04:52.109282 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.109291 2583 generic.go:356] "Generic (PLEG): container finished" podID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerID="2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0" exitCode=0 Jul 13 07:04:52.109282 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.109296 2583 generic.go:356] "Generic (PLEG): container finished" podID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerID="ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465" exitCode=0 Jul 13 07:04:52.109520 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.109323 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerDied","Data":"462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593"} Jul 13 07:04:52.109520 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.109408 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerDied","Data":"82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba"} Jul 13 07:04:52.109520 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.109435 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerDied","Data":"d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a"} Jul 13 07:04:52.109520 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.109458 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerDied","Data":"2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0"} Jul 13 07:04:52.109520 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.109482 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerDied","Data":"ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465"} Jul 13 07:04:52.828696 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.828675 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:52.930401 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930338 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-web\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930401 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930373 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6ht7\" (UniqueName: \"kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-kube-api-access-s6ht7\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930401 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930397 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-web-config\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930414 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930440 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-volume\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930473 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-out\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930489 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-trusted-ca-bundle\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930503 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-cluster-tls-config\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930534 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-metrics-client-ca\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930557 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-tls-assets\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930616 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-main-db\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.930674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930644 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.931116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.930697 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-main-tls\") pod \"b1f264c6-bba6-4e84-8350-8840b43f81a3\" (UID: \"b1f264c6-bba6-4e84-8350-8840b43f81a3\") " Jul 13 07:04:52.931116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.931103 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:52.931704 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.931454 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:52.931704 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.931674 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:04:52.933597 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.933571 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:52.933906 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.933871 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-out" (OuterVolumeSpecName: "config-out") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:04:52.934175 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.934151 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:52.934286 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.934173 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-kube-api-access-s6ht7" (OuterVolumeSpecName: "kube-api-access-s6ht7") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "kube-api-access-s6ht7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:04:52.934445 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.934409 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:52.934514 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.934446 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:04:52.934514 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.934487 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-volume" (OuterVolumeSpecName: "config-volume") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:52.934670 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.934652 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:52.938013 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.937914 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:52.943935 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:52.943910 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-web-config" (OuterVolumeSpecName: "web-config") pod "b1f264c6-bba6-4e84-8350-8840b43f81a3" (UID: "b1f264c6-bba6-4e84-8350-8840b43f81a3"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:53.031953 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.031931 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-main-tls\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.031953 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.031952 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.031962 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6ht7\" (UniqueName: \"kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-kube-api-access-s6ht7\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.031990 2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-web-config\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.031998 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.032014 2583 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-volume\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.032022 2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-config-out\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.032029 2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.032036 2583 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-cluster-tls-config\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.032044 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1f264c6-bba6-4e84-8350-8840b43f81a3-metrics-client-ca\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.032051 2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1f264c6-bba6-4e84-8350-8840b43f81a3-tls-assets\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.032058 2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1f264c6-bba6-4e84-8350-8840b43f81a3-alertmanager-main-db\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.032361 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.032066 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1f264c6-bba6-4e84-8350-8840b43f81a3-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:53.115876 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.115851 2583 generic.go:356] "Generic (PLEG): container finished" podID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerID="a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27" exitCode=0 Jul 13 07:04:53.115963 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.115892 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerDied","Data":"a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27"} Jul 13 07:04:53.115963 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.115929 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.115963 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.115946 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1f264c6-bba6-4e84-8350-8840b43f81a3","Type":"ContainerDied","Data":"631383dd4467c125d31c13a66be011f6fffec9d2176021744a777be6c8153d1d"} Jul 13 07:04:53.116073 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.115978 2583 scope.go:117] "RemoveContainer" containerID="462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593" Jul 13 07:04:53.123255 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.123240 2583 scope.go:117] "RemoveContainer" containerID="82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba" Jul 13 07:04:53.129703 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.129688 2583 scope.go:117] "RemoveContainer" containerID="d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a" Jul 13 07:04:53.135744 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.135729 2583 scope.go:117] "RemoveContainer" containerID="a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27" Jul 13 07:04:53.141936 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.141920 2583 scope.go:117] "RemoveContainer" containerID="2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0" Jul 13 07:04:53.148162 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.148145 2583 scope.go:117] "RemoveContainer" containerID="ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465" Jul 13 07:04:53.154556 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.154539 2583 scope.go:117] "RemoveContainer" containerID="4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd" Jul 13 07:04:53.156609 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.156592 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 07:04:53.163353 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.163338 2583 scope.go:117] "RemoveContainer" containerID="462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593" Jul 13 07:04:53.163651 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:53.163628 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593\": container with ID starting with 462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593 not found: ID does not exist" containerID="462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593" Jul 13 07:04:53.163708 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.163660 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593"} err="failed to get container status \"462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593\": rpc error: code = NotFound desc = could not find container \"462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593\": container with ID starting with 462efcbfd8c3e7b2bd8ad8961a17480b4354bb74dce6eef84a8eb634d1dff593 not found: ID does not exist" Jul 13 07:04:53.163708 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.163682 2583 scope.go:117] "RemoveContainer" containerID="82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba" Jul 13 07:04:53.163974 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:53.163953 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba\": container with ID starting with 82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba not found: ID does not exist" containerID="82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba" Jul 13 07:04:53.164032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.163979 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba"} err="failed to get container status \"82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba\": rpc error: code = NotFound desc = could not find container \"82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba\": container with ID starting with 82e480ba192fc8caf0b158cb5ac7654cf78f98289e25a09a3b36e8ebd5052fba not found: ID does not exist" Jul 13 07:04:53.164032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.163998 2583 scope.go:117] "RemoveContainer" containerID="d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a" Jul 13 07:04:53.164273 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:53.164248 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a\": container with ID starting with d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a not found: ID does not exist" containerID="d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a" Jul 13 07:04:53.164371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.164277 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a"} err="failed to get container status \"d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a\": rpc error: code = NotFound desc = could not find container \"d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a\": container with ID starting with d6a55f68cd4460076cab945a75511f322b44677508408512277fc58244f9b39a not found: ID does not exist" Jul 13 07:04:53.164371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.164292 2583 scope.go:117] "RemoveContainer" containerID="a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27" Jul 13 07:04:53.164513 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:53.164498 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27\": container with ID starting with a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27 not found: ID does not exist" containerID="a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27" Jul 13 07:04:53.164560 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.164516 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27"} err="failed to get container status \"a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27\": rpc error: code = NotFound desc = could not find container \"a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27\": container with ID starting with a8787c222772a2814274d1f094f2617d6b1e9710da72211832255dcc2df24a27 not found: ID does not exist" Jul 13 07:04:53.164560 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.164530 2583 scope.go:117] "RemoveContainer" containerID="2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0" Jul 13 07:04:53.164659 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.164612 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 07:04:53.164705 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:53.164687 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0\": container with ID starting with 2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0 not found: ID does not exist" containerID="2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0" Jul 13 07:04:53.164740 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.164701 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0"} err="failed to get container status \"2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0\": rpc error: code = NotFound desc = could not find container \"2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0\": container with ID starting with 2e6018984a8910aa5bef6928fd1be11c1cdf9bfa9aaa7882e0d1ceb8f624dfc0 not found: ID does not exist" Jul 13 07:04:53.164740 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.164713 2583 scope.go:117] "RemoveContainer" containerID="ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465" Jul 13 07:04:53.164888 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:53.164866 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465\": container with ID starting with ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465 not found: ID does not exist" containerID="ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465" Jul 13 07:04:53.164935 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.164888 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465"} err="failed to get container status \"ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465\": rpc error: code = NotFound desc = could not find container \"ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465\": container with ID starting with ccc1f89d2bd338187a47af975e51e1553d1741461293b6517488d78831d71465 not found: ID does not exist" Jul 13 07:04:53.164935 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.164898 2583 scope.go:117] "RemoveContainer" containerID="4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd" Jul 13 07:04:53.165061 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:53.165048 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd\": container with ID starting with 4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd not found: ID does not exist" containerID="4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd" Jul 13 07:04:53.165102 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.165062 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd"} err="failed to get container status \"4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd\": rpc error: code = NotFound desc = could not find container \"4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd\": container with ID starting with 4f8c253ce6944df7084396ef9d5ef337282a0b6b473bfc3e0d1823c1928be5cd not found: ID does not exist" Jul 13 07:04:53.201881 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.201813 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 07:04:53.202122 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202110 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="config-reloader" Jul 13 07:04:53.202161 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202123 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="config-reloader" Jul 13 07:04:53.202161 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202130 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy-web" Jul 13 07:04:53.202161 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202136 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy-web" Jul 13 07:04:53.202161 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202144 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy-metric" Jul 13 07:04:53.202161 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202149 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy-metric" Jul 13 07:04:53.202161 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202158 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="init-config-reloader" Jul 13 07:04:53.202161 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202163 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="init-config-reloader" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202186 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202191 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202214 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="alertmanager" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202219 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="alertmanager" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202224 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="prom-label-proxy" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202230 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="prom-label-proxy" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202237 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c761caa1-77da-4751-87a9-3ea83266e576" containerName="console" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202243 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c761caa1-77da-4751-87a9-3ea83266e576" containerName="console" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202292 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="prom-label-proxy" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202298 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="alertmanager" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202305 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c761caa1-77da-4751-87a9-3ea83266e576" containerName="console" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202312 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202317 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy-web" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202325 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="config-reloader" Jul 13 07:04:53.202425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.202330 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" containerName="kube-rbac-proxy-metric" Jul 13 07:04:53.206086 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.206069 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.208796 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.208777 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 07:04:53.208901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.208797 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 07:04:53.208901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.208802 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-f42rr\"" Jul 13 07:04:53.208901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.208812 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 07:04:53.209073 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.208924 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 07:04:53.209141 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.209126 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 07:04:53.209211 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.209187 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 07:04:53.209257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.209240 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 07:04:53.209303 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.209257 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 07:04:53.214105 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.214089 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 07:04:53.220981 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.220960 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 07:04:53.334950 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.334926 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.334957 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f020a5ad-4a70-418a-a7bd-667af16bf35d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.334980 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.335017 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f020a5ad-4a70-418a-a7bd-667af16bf35d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.335036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f020a5ad-4a70-418a-a7bd-667af16bf35d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.335049 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxpj4\" (UniqueName: \"kubernetes.io/projected/f020a5ad-4a70-418a-a7bd-667af16bf35d-kube-api-access-pxpj4\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335260 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.335072 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-config-volume\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335260 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.335158 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f020a5ad-4a70-418a-a7bd-667af16bf35d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335260 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.335188 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335260 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.335221 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-web-config\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335260 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.335250 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335418 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.335276 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f020a5ad-4a70-418a-a7bd-667af16bf35d-config-out\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.335418 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.335297 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.426976 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.426957 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1f264c6-bba6-4e84-8350-8840b43f81a3" path="/var/lib/kubelet/pods/b1f264c6-bba6-4e84-8350-8840b43f81a3/volumes" Jul 13 07:04:53.436084 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436068 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436136 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436093 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f020a5ad-4a70-418a-a7bd-667af16bf35d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436136 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436111 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f020a5ad-4a70-418a-a7bd-667af16bf35d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436136 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436125 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxpj4\" (UniqueName: \"kubernetes.io/projected/f020a5ad-4a70-418a-a7bd-667af16bf35d-kube-api-access-pxpj4\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436298 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436151 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-config-volume\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436298 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436178 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f020a5ad-4a70-418a-a7bd-667af16bf35d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436298 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436220 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436298 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436236 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-web-config\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436298 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436257 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436298 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436289 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f020a5ad-4a70-418a-a7bd-667af16bf35d-config-out\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436572 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436323 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436572 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436356 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436572 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436389 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f020a5ad-4a70-418a-a7bd-667af16bf35d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.436774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436753 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f020a5ad-4a70-418a-a7bd-667af16bf35d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.437843 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.436855 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f020a5ad-4a70-418a-a7bd-667af16bf35d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.437843 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.437743 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f020a5ad-4a70-418a-a7bd-667af16bf35d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.439066 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.439045 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f020a5ad-4a70-418a-a7bd-667af16bf35d-config-out\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.439165 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.439077 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f020a5ad-4a70-418a-a7bd-667af16bf35d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.439165 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.439106 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-config-volume\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.439311 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.439258 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.439500 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.439483 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.439579 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.439546 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.439951 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.439932 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.440227 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.440183 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-web-config\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.440463 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.440448 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f020a5ad-4a70-418a-a7bd-667af16bf35d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.444390 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.444373 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxpj4\" (UniqueName: \"kubernetes.io/projected/f020a5ad-4a70-418a-a7bd-667af16bf35d-kube-api-access-pxpj4\") pod \"alertmanager-main-0\" (UID: \"f020a5ad-4a70-418a-a7bd-667af16bf35d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.515780 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.515762 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 07:04:53.644160 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:53.644104 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 07:04:53.646387 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:04:53.646359 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf020a5ad_4a70_418a_a7bd_667af16bf35d.slice/crio-483dba237c633c827d27c26a3ff4028873adaef60a336fb9ba3625f108001457 WatchSource:0}: Error finding container 483dba237c633c827d27c26a3ff4028873adaef60a336fb9ba3625f108001457: Status 404 returned error can't find the container with id 483dba237c633c827d27c26a3ff4028873adaef60a336fb9ba3625f108001457 Jul 13 07:04:54.126499 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:54.126466 2583 generic.go:356] "Generic (PLEG): container finished" podID="f020a5ad-4a70-418a-a7bd-667af16bf35d" containerID="2104b1a3c90893ef575d65d64f41d7a70c2ef89a29f47e1ae2937551152ba8d8" exitCode=0 Jul 13 07:04:54.126882 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:54.126552 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f020a5ad-4a70-418a-a7bd-667af16bf35d","Type":"ContainerDied","Data":"2104b1a3c90893ef575d65d64f41d7a70c2ef89a29f47e1ae2937551152ba8d8"} Jul 13 07:04:54.126882 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:54.126646 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f020a5ad-4a70-418a-a7bd-667af16bf35d","Type":"ContainerStarted","Data":"483dba237c633c827d27c26a3ff4028873adaef60a336fb9ba3625f108001457"} Jul 13 07:04:55.133564 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.133533 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f020a5ad-4a70-418a-a7bd-667af16bf35d","Type":"ContainerStarted","Data":"25a09d6fb7c1a3cb545dddda787589cf01adfd0cf7c61318ba30274bb74562df"} Jul 13 07:04:55.133919 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.133595 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f020a5ad-4a70-418a-a7bd-667af16bf35d","Type":"ContainerStarted","Data":"f864e7e289252167829889c092b9a40f784605d93333eeadfcdeabab79c084e6"} Jul 13 07:04:55.133919 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.133622 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f020a5ad-4a70-418a-a7bd-667af16bf35d","Type":"ContainerStarted","Data":"10e6594449dfaf0aa973fcaebee816a683ec9207f1e2ec7108080f8433cfd4fb"} Jul 13 07:04:55.133919 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.133646 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f020a5ad-4a70-418a-a7bd-667af16bf35d","Type":"ContainerStarted","Data":"9125b391b9277c9e6b92ee81ecf93eff80f37660ff2e9d43456bd2734335abbd"} Jul 13 07:04:55.133919 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.133668 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f020a5ad-4a70-418a-a7bd-667af16bf35d","Type":"ContainerStarted","Data":"31c091ba485f918b7c74e5e62af00d063f741d9dd60b29c9f0f314293d0f4842"} Jul 13 07:04:55.133919 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.133692 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f020a5ad-4a70-418a-a7bd-667af16bf35d","Type":"ContainerStarted","Data":"106eb845f0f25b037d7b8872341e2474643554d7e8eab0db68fbaf713d4aa8a1"} Jul 13 07:04:55.163429 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.163385 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.151238922 podStartE2EDuration="2.163364671s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.125749ms" isStatefulPod=false podCreationTimestamp="2026-07-13 07:04:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:04:55.163061245 +0000 UTC m=+148.337565072" watchObservedRunningTime="2026-07-13 07:04:55.163364671 +0000 UTC m=+148.337868479" Jul 13 07:04:55.814416 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.814383 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 07:04:55.814665 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.814643 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="prometheus" containerID="cri-o://d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c" gracePeriod=600 Jul 13 07:04:55.814742 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.814682 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy" containerID="cri-o://2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897" gracePeriod=600 Jul 13 07:04:55.814794 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.814709 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="thanos-sidecar" containerID="cri-o://91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a" gracePeriod=600 Jul 13 07:04:55.814794 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.814749 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy-thanos" containerID="cri-o://67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a" gracePeriod=600 Jul 13 07:04:55.814794 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.814741 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="config-reloader" containerID="cri-o://561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822" gracePeriod=600 Jul 13 07:04:55.814931 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:55.814697 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy-web" containerID="cri-o://18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e" gracePeriod=600 Jul 13 07:04:56.056825 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.056636 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.141764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141665 2583 generic.go:356] "Generic (PLEG): container finished" podID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerID="67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a" exitCode=0 Jul 13 07:04:56.141764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141691 2583 generic.go:356] "Generic (PLEG): container finished" podID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerID="2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897" exitCode=0 Jul 13 07:04:56.141764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141697 2583 generic.go:356] "Generic (PLEG): container finished" podID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerID="18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e" exitCode=0 Jul 13 07:04:56.141764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141702 2583 generic.go:356] "Generic (PLEG): container finished" podID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerID="91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a" exitCode=0 Jul 13 07:04:56.141764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141695 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerDied","Data":"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a"} Jul 13 07:04:56.141764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141763 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.142501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141794 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerDied","Data":"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897"} Jul 13 07:04:56.142501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141707 2583 generic.go:356] "Generic (PLEG): container finished" podID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerID="561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822" exitCode=0 Jul 13 07:04:56.142501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141807 2583 generic.go:356] "Generic (PLEG): container finished" podID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerID="d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c" exitCode=0 Jul 13 07:04:56.142501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141833 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerDied","Data":"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e"} Jul 13 07:04:56.142501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141858 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerDied","Data":"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a"} Jul 13 07:04:56.142501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141891 2583 scope.go:117] "RemoveContainer" containerID="67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a" Jul 13 07:04:56.142501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.141894 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerDied","Data":"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822"} Jul 13 07:04:56.142501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.142013 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerDied","Data":"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c"} Jul 13 07:04:56.142501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.142042 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6d23102-346e-4d51-aa07-bd4632ebe863","Type":"ContainerDied","Data":"b57488f68498e5a04b4d16634659471d01a870f869f25bf408f4aa678b1f602b"} Jul 13 07:04:56.150036 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.150022 2583 scope.go:117] "RemoveContainer" containerID="2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897" Jul 13 07:04:56.156071 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156050 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156169 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156078 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-config-out\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156169 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156111 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-2\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156169 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156140 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-thanos-prometheus-http-client-file\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156345 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156179 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-trusted-ca-bundle\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156345 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156243 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-metrics-client-certs\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156345 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156278 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-tls\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156345 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156303 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s2f8\" (UniqueName: \"kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-kube-api-access-2s2f8\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156345 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156341 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-metrics-client-ca\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156577 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156377 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-config\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156577 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156404 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-0\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156577 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156427 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-serving-certs-ca-bundle\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156577 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156436 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:56.156577 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156461 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-db\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156577 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156508 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-kubelet-serving-ca-bundle\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156577 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156554 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-grpc-tls\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156962 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156580 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156962 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156584 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:56.156962 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156604 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-tls-assets\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156962 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156635 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-kube-rbac-proxy\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156962 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156666 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-1\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156962 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156688 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-web-config\") pod \"c6d23102-346e-4d51-aa07-bd4632ebe863\" (UID: \"c6d23102-346e-4d51-aa07-bd4632ebe863\") " Jul 13 07:04:56.156962 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156958 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.157312 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156968 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:56.157312 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.156977 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.157464 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.157358 2583 scope.go:117] "RemoveContainer" containerID="18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e" Jul 13 07:04:56.157941 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.157678 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:56.157941 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.157867 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:56.159662 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.159634 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:56.159757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.159732 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:04:56.160671 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.160427 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:56.160671 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.160641 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-config-out" (OuterVolumeSpecName: "config-out") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:04:56.160968 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.160944 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:56.160968 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.160954 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-kube-api-access-2s2f8" (OuterVolumeSpecName: "kube-api-access-2s2f8") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "kube-api-access-2s2f8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:04:56.161107 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.161088 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:56.161462 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.161407 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-config" (OuterVolumeSpecName: "config") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:56.165236 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.161728 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:56.165236 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.161833 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:56.165236 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.161862 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:56.165236 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.161946 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:56.165236 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.162054 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:56.165236 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.162488 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:04:56.170077 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.170056 2583 scope.go:117] "RemoveContainer" containerID="91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a" Jul 13 07:04:56.175003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.174968 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-web-config" (OuterVolumeSpecName: "web-config") pod "c6d23102-346e-4d51-aa07-bd4632ebe863" (UID: "c6d23102-346e-4d51-aa07-bd4632ebe863"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:56.176853 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.176840 2583 scope.go:117] "RemoveContainer" containerID="561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822" Jul 13 07:04:56.182986 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.182954 2583 scope.go:117] "RemoveContainer" containerID="d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c" Jul 13 07:04:56.189457 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.189442 2583 scope.go:117] "RemoveContainer" containerID="99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca" Jul 13 07:04:56.195619 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.195605 2583 scope.go:117] "RemoveContainer" containerID="67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a" Jul 13 07:04:56.195853 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:56.195836 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": container with ID starting with 67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a not found: ID does not exist" containerID="67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a" Jul 13 07:04:56.195899 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.195860 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a"} err="failed to get container status \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": rpc error: code = NotFound desc = could not find container \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": container with ID starting with 67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a not found: ID does not exist" Jul 13 07:04:56.195899 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.195877 2583 scope.go:117] "RemoveContainer" containerID="2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897" Jul 13 07:04:56.196105 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:56.196088 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": container with ID starting with 2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897 not found: ID does not exist" containerID="2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897" Jul 13 07:04:56.196166 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.196114 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897"} err="failed to get container status \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": rpc error: code = NotFound desc = could not find container \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": container with ID starting with 2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897 not found: ID does not exist" Jul 13 07:04:56.196166 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.196135 2583 scope.go:117] "RemoveContainer" containerID="18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e" Jul 13 07:04:56.196392 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:56.196376 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": container with ID starting with 18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e not found: ID does not exist" containerID="18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e" Jul 13 07:04:56.196446 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.196398 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e"} err="failed to get container status \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": rpc error: code = NotFound desc = could not find container \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": container with ID starting with 18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e not found: ID does not exist" Jul 13 07:04:56.196446 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.196417 2583 scope.go:117] "RemoveContainer" containerID="91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a" Jul 13 07:04:56.196649 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:56.196632 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": container with ID starting with 91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a not found: ID does not exist" containerID="91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a" Jul 13 07:04:56.196704 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.196655 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a"} err="failed to get container status \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": rpc error: code = NotFound desc = could not find container \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": container with ID starting with 91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a not found: ID does not exist" Jul 13 07:04:56.196704 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.196669 2583 scope.go:117] "RemoveContainer" containerID="561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822" Jul 13 07:04:56.196864 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:56.196850 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": container with ID starting with 561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822 not found: ID does not exist" containerID="561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822" Jul 13 07:04:56.196901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.196867 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822"} err="failed to get container status \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": rpc error: code = NotFound desc = could not find container \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": container with ID starting with 561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822 not found: ID does not exist" Jul 13 07:04:56.196901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.196878 2583 scope.go:117] "RemoveContainer" containerID="d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c" Jul 13 07:04:56.197051 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:56.197037 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": container with ID starting with d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c not found: ID does not exist" containerID="d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c" Jul 13 07:04:56.197103 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.197054 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c"} err="failed to get container status \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": rpc error: code = NotFound desc = could not find container \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": container with ID starting with d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c not found: ID does not exist" Jul 13 07:04:56.197103 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.197066 2583 scope.go:117] "RemoveContainer" containerID="99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca" Jul 13 07:04:56.197275 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:04:56.197261 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": container with ID starting with 99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca not found: ID does not exist" containerID="99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca" Jul 13 07:04:56.197324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.197277 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca"} err="failed to get container status \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": rpc error: code = NotFound desc = could not find container \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": container with ID starting with 99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca not found: ID does not exist" Jul 13 07:04:56.197324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.197292 2583 scope.go:117] "RemoveContainer" containerID="67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a" Jul 13 07:04:56.197522 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.197504 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a"} err="failed to get container status \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": rpc error: code = NotFound desc = could not find container \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": container with ID starting with 67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a not found: ID does not exist" Jul 13 07:04:56.197561 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.197524 2583 scope.go:117] "RemoveContainer" containerID="2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897" Jul 13 07:04:56.197723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.197706 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897"} err="failed to get container status \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": rpc error: code = NotFound desc = could not find container \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": container with ID starting with 2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897 not found: ID does not exist" Jul 13 07:04:56.197791 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.197725 2583 scope.go:117] "RemoveContainer" containerID="18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e" Jul 13 07:04:56.197952 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.197931 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e"} err="failed to get container status \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": rpc error: code = NotFound desc = could not find container \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": container with ID starting with 18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e not found: ID does not exist" Jul 13 07:04:56.198021 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.197954 2583 scope.go:117] "RemoveContainer" containerID="91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a" Jul 13 07:04:56.198179 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.198164 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a"} err="failed to get container status \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": rpc error: code = NotFound desc = could not find container \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": container with ID starting with 91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a not found: ID does not exist" Jul 13 07:04:56.198248 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.198178 2583 scope.go:117] "RemoveContainer" containerID="561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822" Jul 13 07:04:56.198434 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.198414 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822"} err="failed to get container status \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": rpc error: code = NotFound desc = could not find container \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": container with ID starting with 561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822 not found: ID does not exist" Jul 13 07:04:56.198484 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.198436 2583 scope.go:117] "RemoveContainer" containerID="d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c" Jul 13 07:04:56.198639 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.198620 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c"} err="failed to get container status \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": rpc error: code = NotFound desc = could not find container \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": container with ID starting with d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c not found: ID does not exist" Jul 13 07:04:56.198707 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.198642 2583 scope.go:117] "RemoveContainer" containerID="99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca" Jul 13 07:04:56.198827 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.198811 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca"} err="failed to get container status \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": rpc error: code = NotFound desc = could not find container \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": container with ID starting with 99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca not found: ID does not exist" Jul 13 07:04:56.198886 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.198828 2583 scope.go:117] "RemoveContainer" containerID="67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a" Jul 13 07:04:56.199056 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.199039 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a"} err="failed to get container status \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": rpc error: code = NotFound desc = could not find container \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": container with ID starting with 67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a not found: ID does not exist" Jul 13 07:04:56.199098 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.199057 2583 scope.go:117] "RemoveContainer" containerID="2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897" Jul 13 07:04:56.199247 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.199231 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897"} err="failed to get container status \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": rpc error: code = NotFound desc = could not find container \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": container with ID starting with 2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897 not found: ID does not exist" Jul 13 07:04:56.199307 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.199249 2583 scope.go:117] "RemoveContainer" containerID="18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e" Jul 13 07:04:56.199474 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.199455 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e"} err="failed to get container status \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": rpc error: code = NotFound desc = could not find container \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": container with ID starting with 18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e not found: ID does not exist" Jul 13 07:04:56.199514 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.199477 2583 scope.go:117] "RemoveContainer" containerID="91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a" Jul 13 07:04:56.199655 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.199638 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a"} err="failed to get container status \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": rpc error: code = NotFound desc = could not find container \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": container with ID starting with 91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a not found: ID does not exist" Jul 13 07:04:56.199698 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.199657 2583 scope.go:117] "RemoveContainer" containerID="561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822" Jul 13 07:04:56.199842 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.199825 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822"} err="failed to get container status \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": rpc error: code = NotFound desc = could not find container \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": container with ID starting with 561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822 not found: ID does not exist" Jul 13 07:04:56.199905 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.199844 2583 scope.go:117] "RemoveContainer" containerID="d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c" Jul 13 07:04:56.200058 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.200043 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c"} err="failed to get container status \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": rpc error: code = NotFound desc = could not find container \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": container with ID starting with d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c not found: ID does not exist" Jul 13 07:04:56.200103 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.200058 2583 scope.go:117] "RemoveContainer" containerID="99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca" Jul 13 07:04:56.200273 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.200256 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca"} err="failed to get container status \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": rpc error: code = NotFound desc = could not find container \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": container with ID starting with 99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca not found: ID does not exist" Jul 13 07:04:56.200342 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.200274 2583 scope.go:117] "RemoveContainer" containerID="67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a" Jul 13 07:04:56.200501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.200485 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a"} err="failed to get container status \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": rpc error: code = NotFound desc = could not find container \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": container with ID starting with 67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a not found: ID does not exist" Jul 13 07:04:56.200546 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.200501 2583 scope.go:117] "RemoveContainer" containerID="2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897" Jul 13 07:04:56.200673 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.200655 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897"} err="failed to get container status \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": rpc error: code = NotFound desc = could not find container \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": container with ID starting with 2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897 not found: ID does not exist" Jul 13 07:04:56.200740 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.200675 2583 scope.go:117] "RemoveContainer" containerID="18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e" Jul 13 07:04:56.200881 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.200866 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e"} err="failed to get container status \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": rpc error: code = NotFound desc = could not find container \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": container with ID starting with 18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e not found: ID does not exist" Jul 13 07:04:56.200920 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.200882 2583 scope.go:117] "RemoveContainer" containerID="91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a" Jul 13 07:04:56.201058 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.201042 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a"} err="failed to get container status \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": rpc error: code = NotFound desc = could not find container \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": container with ID starting with 91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a not found: ID does not exist" Jul 13 07:04:56.201121 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.201059 2583 scope.go:117] "RemoveContainer" containerID="561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822" Jul 13 07:04:56.201269 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.201251 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822"} err="failed to get container status \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": rpc error: code = NotFound desc = could not find container \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": container with ID starting with 561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822 not found: ID does not exist" Jul 13 07:04:56.201269 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.201268 2583 scope.go:117] "RemoveContainer" containerID="d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c" Jul 13 07:04:56.201503 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.201485 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c"} err="failed to get container status \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": rpc error: code = NotFound desc = could not find container \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": container with ID starting with d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c not found: ID does not exist" Jul 13 07:04:56.201552 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.201505 2583 scope.go:117] "RemoveContainer" containerID="99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca" Jul 13 07:04:56.201691 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.201673 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca"} err="failed to get container status \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": rpc error: code = NotFound desc = could not find container \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": container with ID starting with 99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca not found: ID does not exist" Jul 13 07:04:56.201759 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.201693 2583 scope.go:117] "RemoveContainer" containerID="67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a" Jul 13 07:04:56.201914 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.201892 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a"} err="failed to get container status \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": rpc error: code = NotFound desc = could not find container \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": container with ID starting with 67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a not found: ID does not exist" Jul 13 07:04:56.201961 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.201918 2583 scope.go:117] "RemoveContainer" containerID="2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897" Jul 13 07:04:56.202102 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.202085 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897"} err="failed to get container status \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": rpc error: code = NotFound desc = could not find container \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": container with ID starting with 2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897 not found: ID does not exist" Jul 13 07:04:56.202168 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.202103 2583 scope.go:117] "RemoveContainer" containerID="18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e" Jul 13 07:04:56.202360 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.202341 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e"} err="failed to get container status \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": rpc error: code = NotFound desc = could not find container \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": container with ID starting with 18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e not found: ID does not exist" Jul 13 07:04:56.202414 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.202362 2583 scope.go:117] "RemoveContainer" containerID="91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a" Jul 13 07:04:56.202558 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.202537 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a"} err="failed to get container status \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": rpc error: code = NotFound desc = could not find container \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": container with ID starting with 91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a not found: ID does not exist" Jul 13 07:04:56.202558 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.202557 2583 scope.go:117] "RemoveContainer" containerID="561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822" Jul 13 07:04:56.202769 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.202755 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822"} err="failed to get container status \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": rpc error: code = NotFound desc = could not find container \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": container with ID starting with 561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822 not found: ID does not exist" Jul 13 07:04:56.202769 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.202769 2583 scope.go:117] "RemoveContainer" containerID="d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c" Jul 13 07:04:56.202974 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.202948 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c"} err="failed to get container status \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": rpc error: code = NotFound desc = could not find container \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": container with ID starting with d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c not found: ID does not exist" Jul 13 07:04:56.203028 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.202974 2583 scope.go:117] "RemoveContainer" containerID="99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca" Jul 13 07:04:56.203127 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.203112 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca"} err="failed to get container status \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": rpc error: code = NotFound desc = could not find container \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": container with ID starting with 99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca not found: ID does not exist" Jul 13 07:04:56.203127 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.203127 2583 scope.go:117] "RemoveContainer" containerID="67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a" Jul 13 07:04:56.203345 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.203328 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a"} err="failed to get container status \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": rpc error: code = NotFound desc = could not find container \"67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a\": container with ID starting with 67a4451fee8728c189ab7e1adadee4576c0ccfd3d8a09288d465233083bca91a not found: ID does not exist" Jul 13 07:04:56.203419 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.203347 2583 scope.go:117] "RemoveContainer" containerID="2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897" Jul 13 07:04:56.203560 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.203544 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897"} err="failed to get container status \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": rpc error: code = NotFound desc = could not find container \"2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897\": container with ID starting with 2deef21dc6dca78ce3b95bd87d0c53f55545e66a7fcce2d4d509243d03e9e897 not found: ID does not exist" Jul 13 07:04:56.203606 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.203561 2583 scope.go:117] "RemoveContainer" containerID="18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e" Jul 13 07:04:56.203743 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.203726 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e"} err="failed to get container status \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": rpc error: code = NotFound desc = could not find container \"18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e\": container with ID starting with 18ca7e468d260a92031c1efa451c732241ce92f53c13b3e78bcaa9bacb521b7e not found: ID does not exist" Jul 13 07:04:56.203779 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.203744 2583 scope.go:117] "RemoveContainer" containerID="91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a" Jul 13 07:04:56.203962 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.203945 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a"} err="failed to get container status \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": rpc error: code = NotFound desc = could not find container \"91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a\": container with ID starting with 91ad81f257abf816577f57335bc530734cca9ea4c01b357465c44ef5a26b7f7a not found: ID does not exist" Jul 13 07:04:56.203997 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.203963 2583 scope.go:117] "RemoveContainer" containerID="561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822" Jul 13 07:04:56.204153 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.204139 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822"} err="failed to get container status \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": rpc error: code = NotFound desc = could not find container \"561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822\": container with ID starting with 561df1fcf0d4b38f317e52608820a6fa18b7fcaedb0e7d03d83289d067061822 not found: ID does not exist" Jul 13 07:04:56.204214 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.204153 2583 scope.go:117] "RemoveContainer" containerID="d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c" Jul 13 07:04:56.204335 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.204321 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c"} err="failed to get container status \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": rpc error: code = NotFound desc = could not find container \"d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c\": container with ID starting with d9c35f399c0c2d8a40f1cc6dd83888edab588ac267e29e105b6eaa23e602fe8c not found: ID does not exist" Jul 13 07:04:56.204377 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.204335 2583 scope.go:117] "RemoveContainer" containerID="99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca" Jul 13 07:04:56.204515 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.204490 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca"} err="failed to get container status \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": rpc error: code = NotFound desc = could not find container \"99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca\": container with ID starting with 99f15c1f85106b0a3ca11f2dd548b8d0585408f76a8ad3b36c15a8ae6895c2ca not found: ID does not exist" Jul 13 07:04:56.257924 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.257903 2583 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-metrics-client-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.257928 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.257943 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s2f8\" (UniqueName: \"kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-kube-api-access-2s2f8\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.257960 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-metrics-client-ca\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.257974 2583 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-config\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.257989 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258004 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258018 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-db\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258032 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258045 2583 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-grpc-tls\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258055 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258063 2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6d23102-346e-4d51-aa07-bd4632ebe863-tls-assets\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258072 2583 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-kube-rbac-proxy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258079 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6d23102-346e-4d51-aa07-bd4632ebe863-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258087 2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-web-config\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258095 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258103 2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6d23102-346e-4d51-aa07-bd4632ebe863-config-out\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.258324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.258111 2583 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6d23102-346e-4d51-aa07-bd4632ebe863-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:04:56.472033 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.472011 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 07:04:56.479368 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.479351 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 07:04:56.503564 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503541 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 07:04:56.503812 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503801 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="config-reloader" Jul 13 07:04:56.503855 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503813 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="config-reloader" Jul 13 07:04:56.503855 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503826 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy" Jul 13 07:04:56.503855 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503831 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy" Jul 13 07:04:56.503855 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503839 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="prometheus" Jul 13 07:04:56.503855 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503845 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="prometheus" Jul 13 07:04:56.503855 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503852 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="thanos-sidecar" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503857 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="thanos-sidecar" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503868 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="init-config-reloader" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503873 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="init-config-reloader" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503882 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy-thanos" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503888 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy-thanos" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503895 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy-web" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503899 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy-web" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503939 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="thanos-sidecar" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503949 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy-web" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503955 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503960 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="kube-rbac-proxy-thanos" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503968 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="config-reloader" Jul 13 07:04:56.504069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.503973 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" containerName="prometheus" Jul 13 07:04:56.507636 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.507620 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.511551 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.511248 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 13 07:04:56.511551 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.511498 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 13 07:04:56.511832 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.511725 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 13 07:04:56.511832 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.511797 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 13 07:04:56.512292 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.511802 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 13 07:04:56.512292 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.511870 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 13 07:04:56.512292 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.511891 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 13 07:04:56.512292 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.511910 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 13 07:04:56.512292 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.511954 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 13 07:04:56.512292 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.512171 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 13 07:04:56.512563 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.512289 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 13 07:04:56.512730 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.512710 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cm7rf32q20esh\"" Jul 13 07:04:56.512888 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.512873 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 13 07:04:56.513003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.512990 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kzqvw\"" Jul 13 07:04:56.513738 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.513712 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 13 07:04:56.516224 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.516182 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 13 07:04:56.531252 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.531230 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 07:04:56.660717 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.660693 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.660857 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.660724 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.660857 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.660742 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.660857 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.660759 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.660857 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.660799 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.660857 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.660846 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9mlp\" (UniqueName: \"kubernetes.io/projected/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-kube-api-access-m9mlp\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.660874 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.660900 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.660937 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-config\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.660971 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.661038 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.661065 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.661097 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661357 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.661113 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661357 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.661132 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661357 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.661148 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661357 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.661164 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661357 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.661177 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661357 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.661191 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.661357 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.661224 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.761960 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.761936 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.761963 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.761983 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762001 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762028 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762054 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762081 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762104 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762125 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762180 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762232 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762257 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.762291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762283 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762304 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762339 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9mlp\" (UniqueName: \"kubernetes.io/projected/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-kube-api-access-m9mlp\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762372 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762402 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-config\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762465 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762623 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.762677 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.763010 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763832 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.763676 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.763947 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.763923 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.764153 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.764129 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.764153 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.764144 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.766452 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.766196 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.766452 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.766348 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.766452 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.766421 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-config\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.766821 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.766779 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.766821 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.766788 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.767011 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.766927 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.767011 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.766959 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.767124 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.767089 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.767124 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.767096 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.767238 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.767134 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.767482 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.767466 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.768004 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.767987 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.772408 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.772392 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9mlp\" (UniqueName: \"kubernetes.io/projected/5bbd6c48-eb6d-4ef6-ac41-981999cd95fe-kube-api-access-m9mlp\") pod \"prometheus-k8s-0\" (UID: \"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.819327 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.819306 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:04:56.956154 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:56.956121 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 07:04:56.959573 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:04:56.959550 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bbd6c48_eb6d_4ef6_ac41_981999cd95fe.slice/crio-0dd014de5a78914299af97345aa666bc749f96fdbe9c9fabeb166764ca925c00 WatchSource:0}: Error finding container 0dd014de5a78914299af97345aa666bc749f96fdbe9c9fabeb166764ca925c00: Status 404 returned error can't find the container with id 0dd014de5a78914299af97345aa666bc749f96fdbe9c9fabeb166764ca925c00 Jul 13 07:04:57.148679 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:57.148656 2583 generic.go:356] "Generic (PLEG): container finished" podID="5bbd6c48-eb6d-4ef6-ac41-981999cd95fe" containerID="cbc848badcfeda8c9851db05ce61830fe27154ea1fca2fbe83e4991261a6887c" exitCode=0 Jul 13 07:04:57.149044 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:57.148713 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe","Type":"ContainerDied","Data":"cbc848badcfeda8c9851db05ce61830fe27154ea1fca2fbe83e4991261a6887c"} Jul 13 07:04:57.149044 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:57.148760 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe","Type":"ContainerStarted","Data":"0dd014de5a78914299af97345aa666bc749f96fdbe9c9fabeb166764ca925c00"} Jul 13 07:04:57.430540 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:57.430501 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6d23102-346e-4d51-aa07-bd4632ebe863" path="/var/lib/kubelet/pods/c6d23102-346e-4d51-aa07-bd4632ebe863/volumes" Jul 13 07:04:58.156256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:58.156224 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe","Type":"ContainerStarted","Data":"7f561651a40faa8bb39572943bc2b5e767e43fdb9fc19aa65aa7f680eb374770"} Jul 13 07:04:58.156566 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:58.156290 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe","Type":"ContainerStarted","Data":"5fda86d0b9a9d0a00fa975d6769f1114a8e05c1fe265bb10f9ae5c4aeff7fad2"} Jul 13 07:04:58.156566 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:58.156317 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe","Type":"ContainerStarted","Data":"fa397fe7ecf8c04dd2e958ebe96e54e95b534231c70c761e89c386f25f713ebe"} Jul 13 07:04:58.156566 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:58.156339 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe","Type":"ContainerStarted","Data":"d6ed8ba96efdf76a721f7c3ee4c2ca9e9dcd005c4072c9ed7144f79036c296b3"} Jul 13 07:04:58.156566 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:58.156363 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe","Type":"ContainerStarted","Data":"a55dc97d81d85755ef1c007085595cb0622ad99ebe386e793d9a74e64b7614dd"} Jul 13 07:04:58.156566 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:58.156385 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bbd6c48-eb6d-4ef6-ac41-981999cd95fe","Type":"ContainerStarted","Data":"c4d0a26cf4de4a20c3d149514abb77bc824f7e23eb4a59a16370de5607b3d68b"} Jul 13 07:04:58.183175 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:58.183109 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.1659004 podStartE2EDuration="2.18309301s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.19261ms" isStatefulPod=false podCreationTimestamp="2026-07-13 07:04:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:04:58.180998502 +0000 UTC m=+151.355502308" watchObservedRunningTime="2026-07-13 07:04:58.18309301 +0000 UTC m=+151.357596816" Jul 13 07:04:59.211349 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:59.211322 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:04:59.259997 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:04:59.259963 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:05:00.872918 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:00.872881 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:05:00.926696 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:00.926671 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:05:00.980240 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:00.980221 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:05:01.819627 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:01.819593 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:05:11.449340 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:11.449311 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:05:11.551053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:11.551029 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:05:32.613793 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:32.613746 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:05:32.677912 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:32.677881 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:05:56.820068 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:56.820039 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:05:56.835324 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:56.835298 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:05:57.404304 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:05:57.404273 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 07:06:01.760978 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:01.760938 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:01.863120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:01.863073 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:28.508487 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:28.508446 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:29.285291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:29.285255 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:32.143767 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:32.143732 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:32.249137 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:32.249092 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:33.485138 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:33.485098 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:33.580959 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:33.580919 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:34.295441 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.295398 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:34.393247 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.393185 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:34.593838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.593773 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-g9lk2"] Jul 13 07:06:34.597170 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.597151 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" Jul 13 07:06:34.599846 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.599828 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7xk8k\"" Jul 13 07:06:34.599951 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.599865 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 07:06:34.599951 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.599917 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 07:06:34.605554 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.605533 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-g9lk2"] Jul 13 07:06:34.713392 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.713367 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fe664dc-d579-4b76-b230-523e91abda85-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-g9lk2\" (UID: \"8fe664dc-d579-4b76-b230-523e91abda85\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" Jul 13 07:06:34.713486 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.713426 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtvsr\" (UniqueName: \"kubernetes.io/projected/8fe664dc-d579-4b76-b230-523e91abda85-kube-api-access-gtvsr\") pod \"cert-manager-webhook-6486ffd997-g9lk2\" (UID: \"8fe664dc-d579-4b76-b230-523e91abda85\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" Jul 13 07:06:34.813689 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.813667 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtvsr\" (UniqueName: \"kubernetes.io/projected/8fe664dc-d579-4b76-b230-523e91abda85-kube-api-access-gtvsr\") pod \"cert-manager-webhook-6486ffd997-g9lk2\" (UID: \"8fe664dc-d579-4b76-b230-523e91abda85\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" Jul 13 07:06:34.813779 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.813708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fe664dc-d579-4b76-b230-523e91abda85-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-g9lk2\" (UID: \"8fe664dc-d579-4b76-b230-523e91abda85\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" Jul 13 07:06:34.822417 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.822395 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fe664dc-d579-4b76-b230-523e91abda85-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-g9lk2\" (UID: \"8fe664dc-d579-4b76-b230-523e91abda85\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" Jul 13 07:06:34.822513 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.822472 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtvsr\" (UniqueName: \"kubernetes.io/projected/8fe664dc-d579-4b76-b230-523e91abda85-kube-api-access-gtvsr\") pod \"cert-manager-webhook-6486ffd997-g9lk2\" (UID: \"8fe664dc-d579-4b76-b230-523e91abda85\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" Jul 13 07:06:34.914193 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:34.914138 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" Jul 13 07:06:35.040233 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:35.040183 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-g9lk2"] Jul 13 07:06:35.043047 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:06:35.043020 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fe664dc_d579_4b76_b230_523e91abda85.slice/crio-52c8564bcc084d910697f7abf060aa1847fb0567f6bfc81968b9bf31c2a3c5f7 WatchSource:0}: Error finding container 52c8564bcc084d910697f7abf060aa1847fb0567f6bfc81968b9bf31c2a3c5f7: Status 404 returned error can't find the container with id 52c8564bcc084d910697f7abf060aa1847fb0567f6bfc81968b9bf31c2a3c5f7 Jul 13 07:06:35.534484 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:35.534452 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" event={"ID":"8fe664dc-d579-4b76-b230-523e91abda85","Type":"ContainerStarted","Data":"52c8564bcc084d910697f7abf060aa1847fb0567f6bfc81968b9bf31c2a3c5f7"} Jul 13 07:06:36.492486 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:36.492437 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:36.685155 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:36.685120 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:38.550002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:38.549967 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" event={"ID":"8fe664dc-d579-4b76-b230-523e91abda85","Type":"ContainerStarted","Data":"f21f575094db646b2d4892d2268a5c2de3407cfbd59dd1c9654ca413503f12b9"} Jul 13 07:06:38.550508 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:38.550097 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" Jul 13 07:06:38.566871 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:38.566822 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" podStartSLOduration=1.504873245 podStartE2EDuration="4.566807638s" totalImagesPullingTime="3.061934393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:06:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:06:38.56537336 +0000 UTC m=+251.739877185" watchObservedRunningTime="2026-07-13 07:06:38.566807638 +0000 UTC m=+251.741311444" Jul 13 07:06:38.896759 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:38.896684 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:38.997842 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:38.997811 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:42.150810 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:42.150780 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:42.209248 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:42.209199 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:44.557523 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:44.557481 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-g9lk2" Jul 13 07:06:44.707866 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:44.707833 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:44.921691 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:44.921618 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:45.021216 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:45.021170 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:47.128279 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.128249 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv"] Jul 13 07:06:47.131606 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.131588 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" Jul 13 07:06:47.134399 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.134382 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:06:47.135497 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.135478 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-pn9k2\"" Jul 13 07:06:47.135625 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.135504 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 07:06:47.141772 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.141751 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv"] Jul 13 07:06:47.228363 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.228335 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j6nm\" (UniqueName: \"kubernetes.io/projected/23c9757a-37ba-49de-b3e9-4a50a4f84f6d-kube-api-access-8j6nm\") pod \"openshift-lws-operator-fd8ccff4c-2t9jv\" (UID: \"23c9757a-37ba-49de-b3e9-4a50a4f84f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" Jul 13 07:06:47.228546 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.228380 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23c9757a-37ba-49de-b3e9-4a50a4f84f6d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2t9jv\" (UID: \"23c9757a-37ba-49de-b3e9-4a50a4f84f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" Jul 13 07:06:47.329667 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.329630 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j6nm\" (UniqueName: \"kubernetes.io/projected/23c9757a-37ba-49de-b3e9-4a50a4f84f6d-kube-api-access-8j6nm\") pod \"openshift-lws-operator-fd8ccff4c-2t9jv\" (UID: \"23c9757a-37ba-49de-b3e9-4a50a4f84f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" Jul 13 07:06:47.329833 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.329684 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23c9757a-37ba-49de-b3e9-4a50a4f84f6d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2t9jv\" (UID: \"23c9757a-37ba-49de-b3e9-4a50a4f84f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" Jul 13 07:06:47.330026 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.330009 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23c9757a-37ba-49de-b3e9-4a50a4f84f6d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2t9jv\" (UID: \"23c9757a-37ba-49de-b3e9-4a50a4f84f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" Jul 13 07:06:47.339373 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.339353 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j6nm\" (UniqueName: \"kubernetes.io/projected/23c9757a-37ba-49de-b3e9-4a50a4f84f6d-kube-api-access-8j6nm\") pod \"openshift-lws-operator-fd8ccff4c-2t9jv\" (UID: \"23c9757a-37ba-49de-b3e9-4a50a4f84f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" Jul 13 07:06:47.440584 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.440522 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" Jul 13 07:06:47.566697 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.566675 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv"] Jul 13 07:06:47.568542 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:06:47.568515 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23c9757a_37ba_49de_b3e9_4a50a4f84f6d.slice/crio-689687b2aa263f01efc9ae73ee6b99295e0c514e12733487c0fa736de79af58c WatchSource:0}: Error finding container 689687b2aa263f01efc9ae73ee6b99295e0c514e12733487c0fa736de79af58c: Status 404 returned error can't find the container with id 689687b2aa263f01efc9ae73ee6b99295e0c514e12733487c0fa736de79af58c Jul 13 07:06:47.586447 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:47.586423 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" event={"ID":"23c9757a-37ba-49de-b3e9-4a50a4f84f6d","Type":"ContainerStarted","Data":"689687b2aa263f01efc9ae73ee6b99295e0c514e12733487c0fa736de79af58c"} Jul 13 07:06:50.601220 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:50.601181 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" event={"ID":"23c9757a-37ba-49de-b3e9-4a50a4f84f6d","Type":"ContainerStarted","Data":"62032beb04395d0b0d7fe805d82ce2c2769e6ab2c568d759881c82a7190f7b5b"} Jul 13 07:06:50.618254 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:50.618189 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2t9jv" podStartSLOduration=1.205708317 podStartE2EDuration="3.618174207s" totalImagesPullingTime="2.41246589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:06:50.616459369 +0000 UTC m=+263.790963201" watchObservedRunningTime="2026-07-13 07:06:50.618174207 +0000 UTC m=+263.792678015" Jul 13 07:06:51.368977 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:51.368945 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:51.472590 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:51.472558 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:06:53.454068 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:06:53.454036 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:05.924270 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:05.924240 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:06.014605 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.014576 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:06.511579 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.511555 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:06.744070 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.744041 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:06.854476 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.854412 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:06.919926 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.919905 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5"] Jul 13 07:07:06.926500 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.926480 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:06.931834 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.931817 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 07:07:06.932039 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.932026 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 07:07:06.932130 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.932110 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-28h86\"" Jul 13 07:07:06.932385 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.932369 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 07:07:06.932544 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.932526 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 07:07:06.941048 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.941023 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5"] Jul 13 07:07:06.980167 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.980146 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1bc19e2c-748b-40e2-afef-60e7f005254c-manifests\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:06.980273 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.980178 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1bc19e2c-748b-40e2-afef-60e7f005254c-tmp\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:06.980273 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.980261 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bc19e2c-748b-40e2-afef-60e7f005254c-webhook-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:06.980345 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.980307 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqfg4\" (UniqueName: \"kubernetes.io/projected/1bc19e2c-748b-40e2-afef-60e7f005254c-kube-api-access-qqfg4\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:06.980379 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:06.980360 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1bc19e2c-748b-40e2-afef-60e7f005254c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.034955 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.034933 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:07.081243 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.081220 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1bc19e2c-748b-40e2-afef-60e7f005254c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.081330 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.081256 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1bc19e2c-748b-40e2-afef-60e7f005254c-manifests\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.081330 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.081291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1bc19e2c-748b-40e2-afef-60e7f005254c-tmp\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.081415 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.081339 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bc19e2c-748b-40e2-afef-60e7f005254c-webhook-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.081415 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.081377 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqfg4\" (UniqueName: \"kubernetes.io/projected/1bc19e2c-748b-40e2-afef-60e7f005254c-kube-api-access-qqfg4\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.081758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.081726 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1bc19e2c-748b-40e2-afef-60e7f005254c-manifests\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.083608 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.083582 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1bc19e2c-748b-40e2-afef-60e7f005254c-tmp\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.083844 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.083828 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1bc19e2c-748b-40e2-afef-60e7f005254c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.083889 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.083864 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bc19e2c-748b-40e2-afef-60e7f005254c-webhook-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.097454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.097435 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqfg4\" (UniqueName: \"kubernetes.io/projected/1bc19e2c-748b-40e2-afef-60e7f005254c-kube-api-access-qqfg4\") pod \"opendatahub-operator-controller-manager-7487689bdd-s84l5\" (UID: \"1bc19e2c-748b-40e2-afef-60e7f005254c\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.164360 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.164303 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:07.236678 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.236653 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:07.366523 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.366498 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5"] Jul 13 07:07:07.369128 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:07:07.369102 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc19e2c_748b_40e2_afef_60e7f005254c.slice/crio-d247145c3369844abac28d8d7abe8993d8df09ed08928824732b874b4b69c5ee WatchSource:0}: Error finding container d247145c3369844abac28d8d7abe8993d8df09ed08928824732b874b4b69c5ee: Status 404 returned error can't find the container with id d247145c3369844abac28d8d7abe8993d8df09ed08928824732b874b4b69c5ee Jul 13 07:07:07.670168 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:07.670131 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" event={"ID":"1bc19e2c-748b-40e2-afef-60e7f005254c","Type":"ContainerStarted","Data":"d247145c3369844abac28d8d7abe8993d8df09ed08928824732b874b4b69c5ee"} Jul 13 07:07:08.922097 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:08.922059 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:10.685828 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:10.685752 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" event={"ID":"1bc19e2c-748b-40e2-afef-60e7f005254c","Type":"ContainerStarted","Data":"d6968a2e7c97494889a0500699b29809dab3647dc6afa71ba4fc2bbbf6c53a52"} Jul 13 07:07:11.691855 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:11.691822 2583 generic.go:356] "Generic (PLEG): container finished" podID="1bc19e2c-748b-40e2-afef-60e7f005254c" containerID="d6968a2e7c97494889a0500699b29809dab3647dc6afa71ba4fc2bbbf6c53a52" exitCode=0 Jul 13 07:07:11.692227 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:11.691889 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" event={"ID":"1bc19e2c-748b-40e2-afef-60e7f005254c","Type":"ContainerDied","Data":"d6968a2e7c97494889a0500699b29809dab3647dc6afa71ba4fc2bbbf6c53a52"} Jul 13 07:07:13.702878 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:13.702829 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" event={"ID":"1bc19e2c-748b-40e2-afef-60e7f005254c","Type":"ContainerStarted","Data":"ffe6ad89831d725cb59708ddcbd4c4a91228d4c5f0a39e135cf59b5c1d05be57"} Jul 13 07:07:13.703317 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:13.702962 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:13.726930 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:13.726875 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" podStartSLOduration=2.6330149130000002 podStartE2EDuration="7.726860509s" totalImagesPullingTime="3.998895199s" totalInitContainerRuntime="1.094950397s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:07:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:07:13.72436266 +0000 UTC m=+286.898866486" watchObservedRunningTime="2026-07-13 07:07:13.726860509 +0000 UTC m=+286.901364320" Jul 13 07:07:17.546194 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:17.546159 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:17.655127 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:17.655092 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:18.065836 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:18.065804 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:18.165330 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:18.165298 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:24.708867 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:24.708836 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-s84l5" Jul 13 07:07:26.258344 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:26.258310 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:26.631245 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:26.631130 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:26.731445 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:26.731417 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:26.971726 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:26.971694 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:27.069382 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:27.069343 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:27.177360 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:27.177332 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:27.254231 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:27.254159 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 07:07:27.286040 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:27.286012 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:27.382971 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:27.382939 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:27.538925 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:27.538864 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:27.982677 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:27.982653 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:28.104457 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:28.104429 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:28.210553 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:28.210524 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:28.353353 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:28.353290 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:28.416074 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:28.416040 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:28.981188 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:28.981155 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:34.173022 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.172992 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:34.275277 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.275250 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:34.919698 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.919665 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99"] Jul 13 07:07:34.923860 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.923834 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:34.929101 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.929079 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 07:07:34.929660 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.929628 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ctn4b\"" Jul 13 07:07:34.929774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.929680 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 07:07:34.929774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.929699 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 07:07:34.933100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.933077 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99"] Jul 13 07:07:34.964743 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.964719 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:34.993796 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.993771 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:34.993920 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.993806 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/105d4deb-f1d8-42cc-8851-d3987c1d440b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:34.993920 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.993828 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:34.993920 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.993891 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsrgc\" (UniqueName: \"kubernetes.io/projected/105d4deb-f1d8-42cc-8851-d3987c1d440b-kube-api-access-bsrgc\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:34.994092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.993998 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:34.994092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.994036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:34.994092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.994065 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:34.994319 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.994130 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:34.994319 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:34.994155 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095274 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095244 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095474 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095292 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095474 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095308 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095474 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095331 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095474 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095351 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/105d4deb-f1d8-42cc-8851-d3987c1d440b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095474 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095368 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095474 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095387 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsrgc\" (UniqueName: \"kubernetes.io/projected/105d4deb-f1d8-42cc-8851-d3987c1d440b-kube-api-access-bsrgc\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095474 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095423 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095806 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095635 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095806 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095702 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095806 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095754 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.095930 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.095894 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.096105 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.096083 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.096224 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.096187 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/105d4deb-f1d8-42cc-8851-d3987c1d440b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.097680 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.097658 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.097842 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.097820 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.107179 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.107155 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/105d4deb-f1d8-42cc-8851-d3987c1d440b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.108555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.108533 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsrgc\" (UniqueName: \"kubernetes.io/projected/105d4deb-f1d8-42cc-8851-d3987c1d440b-kube-api-access-bsrgc\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnf99\" (UID: \"105d4deb-f1d8-42cc-8851-d3987c1d440b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.234762 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.234735 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:35.245981 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.245956 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:35.368034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.368005 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99"] Jul 13 07:07:35.370486 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:07:35.370457 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod105d4deb_f1d8_42cc_8851_d3987c1d440b.slice/crio-70a568a4ac180ea0173df715dce97218979f1ac98ab813380d427cb183e2fdb8 WatchSource:0}: Error finding container 70a568a4ac180ea0173df715dce97218979f1ac98ab813380d427cb183e2fdb8: Status 404 returned error can't find the container with id 70a568a4ac180ea0173df715dce97218979f1ac98ab813380d427cb183e2fdb8 Jul 13 07:07:35.374923 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.374904 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:07:35.448371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.448346 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql"] Jul 13 07:07:35.451045 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.451022 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.453801 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.453782 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 07:07:35.453801 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.453797 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 07:07:35.453974 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.453956 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 07:07:35.454016 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.453985 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fgppt\"" Jul 13 07:07:35.465196 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.465175 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql"] Jul 13 07:07:35.508441 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.508366 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:35.601220 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.601176 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8191fc53-4ea3-4812-9a74-645682c6b210-cert\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.601316 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.601226 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8191fc53-4ea3-4812-9a74-645682c6b210-metrics-cert\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.601316 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.601285 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8191fc53-4ea3-4812-9a74-645682c6b210-manager-config\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.601316 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.601307 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sj25\" (UniqueName: \"kubernetes.io/projected/8191fc53-4ea3-4812-9a74-645682c6b210-kube-api-access-5sj25\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.702094 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.702066 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8191fc53-4ea3-4812-9a74-645682c6b210-metrics-cert\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.702222 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.702103 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8191fc53-4ea3-4812-9a74-645682c6b210-manager-config\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.702222 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.702123 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sj25\" (UniqueName: \"kubernetes.io/projected/8191fc53-4ea3-4812-9a74-645682c6b210-kube-api-access-5sj25\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.702222 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.702183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8191fc53-4ea3-4812-9a74-645682c6b210-cert\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.702841 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.702812 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8191fc53-4ea3-4812-9a74-645682c6b210-manager-config\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.704599 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.704578 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8191fc53-4ea3-4812-9a74-645682c6b210-cert\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.704710 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.704663 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8191fc53-4ea3-4812-9a74-645682c6b210-metrics-cert\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.710110 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.710093 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sj25\" (UniqueName: \"kubernetes.io/projected/8191fc53-4ea3-4812-9a74-645682c6b210-kube-api-access-5sj25\") pod \"lws-controller-manager-9f45c7f89-brmql\" (UID: \"8191fc53-4ea3-4812-9a74-645682c6b210\") " pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.761138 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.761093 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:35.805596 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.805559 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" event={"ID":"105d4deb-f1d8-42cc-8851-d3987c1d440b","Type":"ContainerStarted","Data":"70a568a4ac180ea0173df715dce97218979f1ac98ab813380d427cb183e2fdb8"} Jul 13 07:07:35.933017 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:35.932985 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql"] Jul 13 07:07:35.935449 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:07:35.935416 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8191fc53_4ea3_4812_9a74_645682c6b210.slice/crio-199104bc33a2d75d4a0b92bb0398c9b774f3d063a89e871c6437439777733a4d WatchSource:0}: Error finding container 199104bc33a2d75d4a0b92bb0398c9b774f3d063a89e871c6437439777733a4d: Status 404 returned error can't find the container with id 199104bc33a2d75d4a0b92bb0398c9b774f3d063a89e871c6437439777733a4d Jul 13 07:07:36.817103 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:36.815052 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" event={"ID":"8191fc53-4ea3-4812-9a74-645682c6b210","Type":"ContainerStarted","Data":"199104bc33a2d75d4a0b92bb0398c9b774f3d063a89e871c6437439777733a4d"} Jul 13 07:07:37.259501 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.259437 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:37.317346 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.317312 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4q48r"] Jul 13 07:07:37.319840 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.319818 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" Jul 13 07:07:37.322465 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.322444 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 13 07:07:37.322586 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.322505 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bcgfr\"" Jul 13 07:07:37.332657 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.332637 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4q48r"] Jul 13 07:07:37.366537 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.366510 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:37.418396 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.418366 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v7p9\" (UniqueName: \"kubernetes.io/projected/87f77770-a3ed-47b1-9cff-27f628d8b85a-kube-api-access-6v7p9\") pod \"kserve-controller-manager-d6f4b65b8-4q48r\" (UID: \"87f77770-a3ed-47b1-9cff-27f628d8b85a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" Jul 13 07:07:37.418893 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.418433 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87f77770-a3ed-47b1-9cff-27f628d8b85a-cert\") pod \"kserve-controller-manager-d6f4b65b8-4q48r\" (UID: \"87f77770-a3ed-47b1-9cff-27f628d8b85a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" Jul 13 07:07:37.519305 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.519231 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7p9\" (UniqueName: \"kubernetes.io/projected/87f77770-a3ed-47b1-9cff-27f628d8b85a-kube-api-access-6v7p9\") pod \"kserve-controller-manager-d6f4b65b8-4q48r\" (UID: \"87f77770-a3ed-47b1-9cff-27f628d8b85a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" Jul 13 07:07:37.519478 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.519303 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87f77770-a3ed-47b1-9cff-27f628d8b85a-cert\") pod \"kserve-controller-manager-d6f4b65b8-4q48r\" (UID: \"87f77770-a3ed-47b1-9cff-27f628d8b85a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" Jul 13 07:07:37.522219 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.522179 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87f77770-a3ed-47b1-9cff-27f628d8b85a-cert\") pod \"kserve-controller-manager-d6f4b65b8-4q48r\" (UID: \"87f77770-a3ed-47b1-9cff-27f628d8b85a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" Jul 13 07:07:37.550881 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.550857 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v7p9\" (UniqueName: \"kubernetes.io/projected/87f77770-a3ed-47b1-9cff-27f628d8b85a-kube-api-access-6v7p9\") pod \"kserve-controller-manager-d6f4b65b8-4q48r\" (UID: \"87f77770-a3ed-47b1-9cff-27f628d8b85a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" Jul 13 07:07:37.595017 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.594939 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:37.643951 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:37.643921 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" Jul 13 07:07:38.125757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.125716 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:38.247668 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.247617 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:07:38.247782 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.247704 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:07:38.247782 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.247749 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:07:38.379720 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.379640 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4q48r"] Jul 13 07:07:38.383177 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:07:38.383139 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f77770_a3ed_47b1_9cff_27f628d8b85a.slice/crio-1d91217dd524149188870d82c1095704fc44c8782ccbd3c45d4dfba57eb231ce WatchSource:0}: Error finding container 1d91217dd524149188870d82c1095704fc44c8782ccbd3c45d4dfba57eb231ce: Status 404 returned error can't find the container with id 1d91217dd524149188870d82c1095704fc44c8782ccbd3c45d4dfba57eb231ce Jul 13 07:07:38.549051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.549020 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:38.669238 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.669151 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:38.796415 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.796388 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:07:38.826557 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.826529 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" event={"ID":"105d4deb-f1d8-42cc-8851-d3987c1d440b","Type":"ContainerStarted","Data":"e55386975488ee68b14e9475d30d7b7860fb781a3a3113787d2142be5430836e"} Jul 13 07:07:38.827802 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.827778 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" event={"ID":"87f77770-a3ed-47b1-9cff-27f628d8b85a","Type":"ContainerStarted","Data":"1d91217dd524149188870d82c1095704fc44c8782ccbd3c45d4dfba57eb231ce"} Jul 13 07:07:38.829405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.829383 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" event={"ID":"8191fc53-4ea3-4812-9a74-645682c6b210","Type":"ContainerStarted","Data":"f4ede82628845b880c4ad5e36482f4ae342cef1734b4c7edafd3f30aa6858b89"} Jul 13 07:07:38.829496 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.829479 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:07:38.861162 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.861119 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" podStartSLOduration=1.988856689 podStartE2EDuration="4.86110599s" totalImagesPullingTime="2.872249301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:07:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:07:38.859707473 +0000 UTC m=+312.034211282" watchObservedRunningTime="2026-07-13 07:07:38.86110599 +0000 UTC m=+312.035609868" Jul 13 07:07:38.881567 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:38.881524 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" podStartSLOduration=1.570781142 podStartE2EDuration="3.881509836s" totalImagesPullingTime="2.310728694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:07:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:07:38.880512969 +0000 UTC m=+312.055016778" watchObservedRunningTime="2026-07-13 07:07:38.881509836 +0000 UTC m=+312.056013642" Jul 13 07:07:39.235658 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:39.235617 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:39.241097 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:39.241073 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:39.834705 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:39.834673 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:39.835491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:39.835468 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnf99" Jul 13 07:07:41.847535 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:41.847497 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" event={"ID":"87f77770-a3ed-47b1-9cff-27f628d8b85a","Type":"ContainerStarted","Data":"03cdbd1e89d015de1702dcf552d8ed9562248acb11e1482a46aa408e624c0514"} Jul 13 07:07:41.847931 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:41.847578 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" Jul 13 07:07:41.884891 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:41.884844 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" podStartSLOduration=2.238507422 podStartE2EDuration="4.88482792s" totalImagesPullingTime="2.646320498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:07:41.883987711 +0000 UTC m=+315.058491521" watchObservedRunningTime="2026-07-13 07:07:41.88482792 +0000 UTC m=+315.059331734" Jul 13 07:07:49.836065 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:07:49.836037 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-9f45c7f89-brmql" Jul 13 07:08:12.857358 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:12.857330 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4q48r" Jul 13 07:08:27.709217 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:27.709184 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:08:27.806829 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:27.806801 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:08:27.906438 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:27.906409 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:08:29.872962 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:29.872930 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:08:49.230635 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:49.230606 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:08:49.284336 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:49.284309 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:08:56.126570 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:56.126534 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:08:56.233557 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:56.233523 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:08:56.597911 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:56.597877 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:08:59.744000 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:59.743972 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:08:59.843053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:08:59.843013 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:00.152070 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:00.151992 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:00.283081 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:00.283050 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:04.130438 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:04.130406 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:04.772968 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:04.772935 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:06.420106 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:06.420074 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:07.947256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:07.947221 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:08.779444 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:08.779413 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:08.878836 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:08.878805 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:10.181692 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.181659 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw"] Jul 13 07:09:10.185181 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.185161 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw" Jul 13 07:09:10.187774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.187759 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 07:09:10.188778 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.188759 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 13 07:09:10.188834 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.188779 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-stg5s\"" Jul 13 07:09:10.188834 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.188758 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 07:09:10.195410 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.195388 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw"] Jul 13 07:09:10.332725 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.332687 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2s69\" (UniqueName: \"kubernetes.io/projected/d0a15549-9350-4fe4-a515-77f42e19c9d0-kube-api-access-r2s69\") pod \"dns-operator-controller-manager-864dd78c4c-x8vdw\" (UID: \"d0a15549-9350-4fe4-a515-77f42e19c9d0\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw" Jul 13 07:09:10.434287 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.434166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s69\" (UniqueName: \"kubernetes.io/projected/d0a15549-9350-4fe4-a515-77f42e19c9d0-kube-api-access-r2s69\") pod \"dns-operator-controller-manager-864dd78c4c-x8vdw\" (UID: \"d0a15549-9350-4fe4-a515-77f42e19c9d0\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw" Jul 13 07:09:10.443403 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.443379 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2s69\" (UniqueName: \"kubernetes.io/projected/d0a15549-9350-4fe4-a515-77f42e19c9d0-kube-api-access-r2s69\") pod \"dns-operator-controller-manager-864dd78c4c-x8vdw\" (UID: \"d0a15549-9350-4fe4-a515-77f42e19c9d0\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw" Jul 13 07:09:10.495097 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.495061 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw" Jul 13 07:09:10.646473 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.646431 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw"] Jul 13 07:09:10.653536 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:09:10.653506 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a15549_9350_4fe4_a515_77f42e19c9d0.slice/crio-ebeeae2ae45c3f8936c57aa34ad2bc2429ab8679d83fe71aa5537b705ba966a1 WatchSource:0}: Error finding container ebeeae2ae45c3f8936c57aa34ad2bc2429ab8679d83fe71aa5537b705ba966a1: Status 404 returned error can't find the container with id ebeeae2ae45c3f8936c57aa34ad2bc2429ab8679d83fe71aa5537b705ba966a1 Jul 13 07:09:10.761647 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:10.761611 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:11.252988 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:11.252955 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw" event={"ID":"d0a15549-9350-4fe4-a515-77f42e19c9d0","Type":"ContainerStarted","Data":"ebeeae2ae45c3f8936c57aa34ad2bc2429ab8679d83fe71aa5537b705ba966a1"} Jul 13 07:09:12.139427 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:12.139397 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:12.239401 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:12.239368 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:13.264348 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:13.264306 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw" event={"ID":"d0a15549-9350-4fe4-a515-77f42e19c9d0","Type":"ContainerStarted","Data":"4ebf4f4a3a6992445e700d990530f34355d090a382794379e9dc2cbe7aa8885b"} Jul 13 07:09:13.264793 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:13.264386 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw" Jul 13 07:09:13.284327 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:13.284280 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw" podStartSLOduration=0.864338532 podStartE2EDuration="3.28426451s" totalImagesPullingTime="2.419925978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:09:13.283254858 +0000 UTC m=+406.457758684" watchObservedRunningTime="2026-07-13 07:09:13.28426451 +0000 UTC m=+406.458768316" Jul 13 07:09:14.015732 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.015701 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:14.021967 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.021938 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf"] Jul 13 07:09:14.025413 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.025396 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.028028 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.028008 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 13 07:09:14.028130 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.028012 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 13 07:09:14.028390 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.028375 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n6874\"" Jul 13 07:09:14.038615 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.038576 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf"] Jul 13 07:09:14.067673 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.067640 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4164614c-0b92-48e9-a3a3-432a3711ddee-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-ct8bf\" (UID: \"4164614c-0b92-48e9-a3a3-432a3711ddee\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.067673 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.067678 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4164614c-0b92-48e9-a3a3-432a3711ddee-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-ct8bf\" (UID: \"4164614c-0b92-48e9-a3a3-432a3711ddee\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.067886 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.067720 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnptc\" (UniqueName: \"kubernetes.io/projected/4164614c-0b92-48e9-a3a3-432a3711ddee-kube-api-access-tnptc\") pod \"kuadrant-console-plugin-66b5774985-ct8bf\" (UID: \"4164614c-0b92-48e9-a3a3-432a3711ddee\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.168542 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.168508 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4164614c-0b92-48e9-a3a3-432a3711ddee-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-ct8bf\" (UID: \"4164614c-0b92-48e9-a3a3-432a3711ddee\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.168713 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.168548 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4164614c-0b92-48e9-a3a3-432a3711ddee-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-ct8bf\" (UID: \"4164614c-0b92-48e9-a3a3-432a3711ddee\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.168713 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.168573 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnptc\" (UniqueName: \"kubernetes.io/projected/4164614c-0b92-48e9-a3a3-432a3711ddee-kube-api-access-tnptc\") pod \"kuadrant-console-plugin-66b5774985-ct8bf\" (UID: \"4164614c-0b92-48e9-a3a3-432a3711ddee\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.168713 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:09:14.168672 2583 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 13 07:09:14.168908 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:09:14.168751 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4164614c-0b92-48e9-a3a3-432a3711ddee-plugin-serving-cert podName:4164614c-0b92-48e9-a3a3-432a3711ddee nodeName:}" failed. No retries permitted until 2026-07-13 07:09:14.668726643 +0000 UTC m=+407.843230431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/4164614c-0b92-48e9-a3a3-432a3711ddee-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-ct8bf" (UID: "4164614c-0b92-48e9-a3a3-432a3711ddee") : secret "plugin-serving-cert" not found Jul 13 07:09:14.169100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.169083 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4164614c-0b92-48e9-a3a3-432a3711ddee-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-ct8bf\" (UID: \"4164614c-0b92-48e9-a3a3-432a3711ddee\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.177834 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.177812 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnptc\" (UniqueName: \"kubernetes.io/projected/4164614c-0b92-48e9-a3a3-432a3711ddee-kube-api-access-tnptc\") pod \"kuadrant-console-plugin-66b5774985-ct8bf\" (UID: \"4164614c-0b92-48e9-a3a3-432a3711ddee\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.250592 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.250560 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:14.672699 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.672660 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4164614c-0b92-48e9-a3a3-432a3711ddee-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-ct8bf\" (UID: \"4164614c-0b92-48e9-a3a3-432a3711ddee\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.675118 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.675096 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4164614c-0b92-48e9-a3a3-432a3711ddee-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-ct8bf\" (UID: \"4164614c-0b92-48e9-a3a3-432a3711ddee\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:14.934557 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:14.934471 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" Jul 13 07:09:15.061215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:15.061180 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf"] Jul 13 07:09:15.063471 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:09:15.063442 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4164614c_0b92_48e9_a3a3_432a3711ddee.slice/crio-96e98dd01001f60cd069fa4c39074be154f75738e28d8fb3aae692cb3e37626e WatchSource:0}: Error finding container 96e98dd01001f60cd069fa4c39074be154f75738e28d8fb3aae692cb3e37626e: Status 404 returned error can't find the container with id 96e98dd01001f60cd069fa4c39074be154f75738e28d8fb3aae692cb3e37626e Jul 13 07:09:15.274147 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:15.274110 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" event={"ID":"4164614c-0b92-48e9-a3a3-432a3711ddee","Type":"ContainerStarted","Data":"96e98dd01001f60cd069fa4c39074be154f75738e28d8fb3aae692cb3e37626e"} Jul 13 07:09:15.530818 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:15.530746 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:15.586650 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:15.586621 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:23.975439 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:23.975406 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4"] Jul 13 07:09:23.993165 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:23.993108 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4"] Jul 13 07:09:23.993347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:23.993227 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:23.996181 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:23.996160 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rhv55\"" Jul 13 07:09:24.065815 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.065780 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b8b1be7-8027-40e7-a824-5f0c71919d53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5n6m4\" (UID: \"8b8b1be7-8027-40e7-a824-5f0c71919d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:24.065968 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.065836 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7gdz\" (UniqueName: \"kubernetes.io/projected/8b8b1be7-8027-40e7-a824-5f0c71919d53-kube-api-access-w7gdz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5n6m4\" (UID: \"8b8b1be7-8027-40e7-a824-5f0c71919d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:24.157150 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.157086 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:24.166926 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.166896 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b8b1be7-8027-40e7-a824-5f0c71919d53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5n6m4\" (UID: \"8b8b1be7-8027-40e7-a824-5f0c71919d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:24.167071 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.166946 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gdz\" (UniqueName: \"kubernetes.io/projected/8b8b1be7-8027-40e7-a824-5f0c71919d53-kube-api-access-w7gdz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5n6m4\" (UID: \"8b8b1be7-8027-40e7-a824-5f0c71919d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:24.167303 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.167280 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b8b1be7-8027-40e7-a824-5f0c71919d53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5n6m4\" (UID: \"8b8b1be7-8027-40e7-a824-5f0c71919d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:24.175961 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.175936 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7gdz\" (UniqueName: \"kubernetes.io/projected/8b8b1be7-8027-40e7-a824-5f0c71919d53-kube-api-access-w7gdz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5n6m4\" (UID: \"8b8b1be7-8027-40e7-a824-5f0c71919d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:24.271221 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.271129 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x8vdw" Jul 13 07:09:24.274087 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.274055 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:24.306049 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.306024 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:24.700826 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.700743 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4"] Jul 13 07:09:24.710237 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.710154 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4"] Jul 13 07:09:24.725562 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.725514 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq"] Jul 13 07:09:24.737042 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.737014 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:09:24.740738 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.740679 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq"] Jul 13 07:09:24.773949 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.773917 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdde22d6-576c-432e-b6d6-c191985ec14e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2w5cq\" (UID: \"fdde22d6-576c-432e-b6d6-c191985ec14e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:09:24.774109 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.773961 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6lh5\" (UniqueName: \"kubernetes.io/projected/fdde22d6-576c-432e-b6d6-c191985ec14e-kube-api-access-m6lh5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2w5cq\" (UID: \"fdde22d6-576c-432e-b6d6-c191985ec14e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:09:24.875417 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.875369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdde22d6-576c-432e-b6d6-c191985ec14e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2w5cq\" (UID: \"fdde22d6-576c-432e-b6d6-c191985ec14e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:09:24.875602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.875433 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6lh5\" (UniqueName: \"kubernetes.io/projected/fdde22d6-576c-432e-b6d6-c191985ec14e-kube-api-access-m6lh5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2w5cq\" (UID: \"fdde22d6-576c-432e-b6d6-c191985ec14e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:09:24.875819 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.875783 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdde22d6-576c-432e-b6d6-c191985ec14e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2w5cq\" (UID: \"fdde22d6-576c-432e-b6d6-c191985ec14e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:09:24.891443 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:24.891421 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6lh5\" (UniqueName: \"kubernetes.io/projected/fdde22d6-576c-432e-b6d6-c191985ec14e-kube-api-access-m6lh5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2w5cq\" (UID: \"fdde22d6-576c-432e-b6d6-c191985ec14e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:09:25.053484 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:25.053447 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:09:26.601420 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:26.601388 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:26.704256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:26.704221 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:28.749444 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:28.749354 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:30.956373 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:30.956323 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:31.068600 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:31.068567 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:33.143949 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:33.143912 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:33.208180 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:33.208119 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:36.146169 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:36.146145 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq"] Jul 13 07:09:36.147657 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:09:36.147632 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdde22d6_576c_432e_b6d6_c191985ec14e.slice/crio-d23f84e6ba10012dd851fe1f1f00457304d05bdb01f42e7fa31a627bd9061baa WatchSource:0}: Error finding container d23f84e6ba10012dd851fe1f1f00457304d05bdb01f42e7fa31a627bd9061baa: Status 404 returned error can't find the container with id d23f84e6ba10012dd851fe1f1f00457304d05bdb01f42e7fa31a627bd9061baa Jul 13 07:09:36.395792 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:36.395696 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" event={"ID":"4164614c-0b92-48e9-a3a3-432a3711ddee","Type":"ContainerStarted","Data":"fd886ed9aa72d15488fbd6d71e2bd0e46eb108b0ea07de2b9b35c24547a77257"} Jul 13 07:09:36.396896 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:36.396869 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" event={"ID":"fdde22d6-576c-432e-b6d6-c191985ec14e","Type":"ContainerStarted","Data":"d23f84e6ba10012dd851fe1f1f00457304d05bdb01f42e7fa31a627bd9061baa"} Jul 13 07:09:36.411126 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:36.411083 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ct8bf" podStartSLOduration=2.380851731 podStartE2EDuration="23.411069879s" totalImagesPullingTime="21.030218148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:09:36.410566492 +0000 UTC m=+429.585070321" watchObservedRunningTime="2026-07-13 07:09:36.411069879 +0000 UTC m=+429.585573685" Jul 13 07:09:38.548814 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:09:38.548762 2583 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-5n6m4_kuadrant-system_8b8b1be7-8027-40e7-a824-5f0c71919d53_0(c2a92775d9c827f35843b932c0618aca85a19d9e63d9c8b174c3c0fc15111099): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-5n6m4 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 13 07:09:38.549142 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:09:38.548838 2583 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-5n6m4_kuadrant-system_8b8b1be7-8027-40e7-a824-5f0c71919d53_0(c2a92775d9c827f35843b932c0618aca85a19d9e63d9c8b174c3c0fc15111099): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-5n6m4 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:39.417459 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:39.417429 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:39.419784 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:09:39.419753 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-5n6m4\" is forbidden: User \"system:node:ip-10-0-133-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-57.ec2.internal' and this object" podUID="8b8b1be7-8027-40e7-a824-5f0c71919d53" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:39.425904 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:39.425883 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:39.429863 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:09:39.429830 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-5n6m4\" is forbidden: User \"system:node:ip-10-0-133-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-57.ec2.internal' and this object" podUID="8b8b1be7-8027-40e7-a824-5f0c71919d53" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:39.516878 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:39.516849 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7gdz\" (UniqueName: \"kubernetes.io/projected/8b8b1be7-8027-40e7-a824-5f0c71919d53-kube-api-access-w7gdz\") pod \"8b8b1be7-8027-40e7-a824-5f0c71919d53\" (UID: \"8b8b1be7-8027-40e7-a824-5f0c71919d53\") " Jul 13 07:09:39.516989 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:39.516963 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b8b1be7-8027-40e7-a824-5f0c71919d53-extensions-socket-volume\") pod \"8b8b1be7-8027-40e7-a824-5f0c71919d53\" (UID: \"8b8b1be7-8027-40e7-a824-5f0c71919d53\") " Jul 13 07:09:39.517251 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:39.517199 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b8b1be7-8027-40e7-a824-5f0c71919d53-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8b8b1be7-8027-40e7-a824-5f0c71919d53" (UID: "8b8b1be7-8027-40e7-a824-5f0c71919d53"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:09:39.518798 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:39.518778 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b8b1be7-8027-40e7-a824-5f0c71919d53-kube-api-access-w7gdz" (OuterVolumeSpecName: "kube-api-access-w7gdz") pod "8b8b1be7-8027-40e7-a824-5f0c71919d53" (UID: "8b8b1be7-8027-40e7-a824-5f0c71919d53"). InnerVolumeSpecName "kube-api-access-w7gdz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:09:39.618245 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:39.618191 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b8b1be7-8027-40e7-a824-5f0c71919d53-extensions-socket-volume\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:09:39.618245 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:39.618243 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7gdz\" (UniqueName: \"kubernetes.io/projected/8b8b1be7-8027-40e7-a824-5f0c71919d53-kube-api-access-w7gdz\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:09:40.423375 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:40.423340 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:40.426119 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:09:40.426081 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-5n6m4\" is forbidden: User \"system:node:ip-10-0-133-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-57.ec2.internal' and this object" podUID="8b8b1be7-8027-40e7-a824-5f0c71919d53" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:40.433215 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:09:40.433174 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-5n6m4\" is forbidden: User \"system:node:ip-10-0-133-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-57.ec2.internal' and this object" podUID="8b8b1be7-8027-40e7-a824-5f0c71919d53" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5n6m4" Jul 13 07:09:41.428758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:41.428679 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b8b1be7-8027-40e7-a824-5f0c71919d53" path="/var/lib/kubelet/pods/8b8b1be7-8027-40e7-a824-5f0c71919d53/volumes" Jul 13 07:09:41.429898 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:41.429876 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" event={"ID":"fdde22d6-576c-432e-b6d6-c191985ec14e","Type":"ContainerStarted","Data":"0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f"} Jul 13 07:09:41.430031 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:41.430017 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:09:41.449394 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:41.449348 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" podStartSLOduration=12.531307306 podStartE2EDuration="17.449333141s" totalImagesPullingTime="4.918025835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:09:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:09:41.447569838 +0000 UTC m=+434.622073665" watchObservedRunningTime="2026-07-13 07:09:41.449333141 +0000 UTC m=+434.623836947" Jul 13 07:09:41.921941 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:41.921910 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:42.022314 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:42.022284 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:44.372137 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:44.372105 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:44.423783 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:44.423759 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:48.727431 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:48.727400 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:48.828071 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:48.828040 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:52.437063 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:52.437032 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:09:52.553963 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:52.553925 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:52.660922 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:52.660893 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:55.215901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:55.215856 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:09:55.316715 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:09:55.316670 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:02.045546 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:02.045507 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:02.099160 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:02.099116 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:02.545406 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:02.545371 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:02.611224 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:02.611170 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:04.993195 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:04.993149 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:05.049502 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.049463 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:05.290507 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.290420 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq"] Jul 13 07:10:05.290676 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.290643 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" podUID="fdde22d6-576c-432e-b6d6-c191985ec14e" containerName="manager" containerID="cri-o://0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f" gracePeriod=10 Jul 13 07:10:05.536723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.536700 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:10:05.551710 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.551644 2583 generic.go:356] "Generic (PLEG): container finished" podID="fdde22d6-576c-432e-b6d6-c191985ec14e" containerID="0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f" exitCode=0 Jul 13 07:10:05.551710 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.551685 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" Jul 13 07:10:05.551845 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.551724 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" event={"ID":"fdde22d6-576c-432e-b6d6-c191985ec14e","Type":"ContainerDied","Data":"0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f"} Jul 13 07:10:05.551845 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.551774 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq" event={"ID":"fdde22d6-576c-432e-b6d6-c191985ec14e","Type":"ContainerDied","Data":"d23f84e6ba10012dd851fe1f1f00457304d05bdb01f42e7fa31a627bd9061baa"} Jul 13 07:10:05.551845 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.551796 2583 scope.go:117] "RemoveContainer" containerID="0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f" Jul 13 07:10:05.559828 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.559811 2583 scope.go:117] "RemoveContainer" containerID="0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f" Jul 13 07:10:05.560101 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:10:05.560083 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f\": container with ID starting with 0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f not found: ID does not exist" containerID="0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f" Jul 13 07:10:05.560146 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.560109 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f"} err="failed to get container status \"0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f\": rpc error: code = NotFound desc = could not find container \"0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f\": container with ID starting with 0dbea20a81047d76f215eb31f1222bac168d94c879284af52eb01dca0dbd8c1f not found: ID does not exist" Jul 13 07:10:05.643458 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.643425 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6lh5\" (UniqueName: \"kubernetes.io/projected/fdde22d6-576c-432e-b6d6-c191985ec14e-kube-api-access-m6lh5\") pod \"fdde22d6-576c-432e-b6d6-c191985ec14e\" (UID: \"fdde22d6-576c-432e-b6d6-c191985ec14e\") " Jul 13 07:10:05.643624 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.643543 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdde22d6-576c-432e-b6d6-c191985ec14e-extensions-socket-volume\") pod \"fdde22d6-576c-432e-b6d6-c191985ec14e\" (UID: \"fdde22d6-576c-432e-b6d6-c191985ec14e\") " Jul 13 07:10:05.644013 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.643989 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdde22d6-576c-432e-b6d6-c191985ec14e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fdde22d6-576c-432e-b6d6-c191985ec14e" (UID: "fdde22d6-576c-432e-b6d6-c191985ec14e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:10:05.645609 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.645583 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdde22d6-576c-432e-b6d6-c191985ec14e-kube-api-access-m6lh5" (OuterVolumeSpecName: "kube-api-access-m6lh5") pod "fdde22d6-576c-432e-b6d6-c191985ec14e" (UID: "fdde22d6-576c-432e-b6d6-c191985ec14e"). InnerVolumeSpecName "kube-api-access-m6lh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:10:05.745053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.745017 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdde22d6-576c-432e-b6d6-c191985ec14e-extensions-socket-volume\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:10:05.745053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.745045 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6lh5\" (UniqueName: \"kubernetes.io/projected/fdde22d6-576c-432e-b6d6-c191985ec14e-kube-api-access-m6lh5\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:10:05.873404 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.873372 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq"] Jul 13 07:10:05.875304 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:05.875272 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2w5cq"] Jul 13 07:10:07.428580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:07.428543 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdde22d6-576c-432e-b6d6-c191985ec14e" path="/var/lib/kubelet/pods/fdde22d6-576c-432e-b6d6-c191985ec14e/volumes" Jul 13 07:10:08.859953 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.859918 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf"] Jul 13 07:10:08.860398 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.860380 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdde22d6-576c-432e-b6d6-c191985ec14e" containerName="manager" Jul 13 07:10:08.860444 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.860400 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdde22d6-576c-432e-b6d6-c191985ec14e" containerName="manager" Jul 13 07:10:08.860478 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.860454 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdde22d6-576c-432e-b6d6-c191985ec14e" containerName="manager" Jul 13 07:10:08.885288 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.885255 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:08.939090 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.939048 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf"] Jul 13 07:10:08.939274 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.939165 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:08.941788 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.941757 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qnq26\"" Jul 13 07:10:08.973111 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.973080 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:08.973299 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.973121 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:08.973299 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.973153 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:08.973299 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.973218 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:08.973299 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.973241 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:08.973430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.973308 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:08.973430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.973344 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:08.973430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.973366 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:08.973430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:08.973380 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vccg6\" (UniqueName: \"kubernetes.io/projected/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-kube-api-access-vccg6\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074445 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074411 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074453 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074484 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074504 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074521 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074570 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074595 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074620 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074988 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074646 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vccg6\" (UniqueName: \"kubernetes.io/projected/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-kube-api-access-vccg6\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074988 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074887 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.074988 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.074923 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.075221 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.075174 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.075333 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.075313 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.075414 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.075392 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.076936 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.076914 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.077002 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.076991 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.083447 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.083418 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vccg6\" (UniqueName: \"kubernetes.io/projected/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-kube-api-access-vccg6\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.083546 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.083524 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ppflf\" (UID: \"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.161326 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.161233 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:09.249295 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.249257 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:09.407849 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.407819 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf"] Jul 13 07:10:09.410319 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:10:09.410276 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8dd9b2e_20ab_4088_89d2_a674fd3a1aa2.slice/crio-5129e0f84a223aec49ac5a529aaedbec6818b1155670ad9b1745d76c07a79748 WatchSource:0}: Error finding container 5129e0f84a223aec49ac5a529aaedbec6818b1155670ad9b1745d76c07a79748: Status 404 returned error can't find the container with id 5129e0f84a223aec49ac5a529aaedbec6818b1155670ad9b1745d76c07a79748 Jul 13 07:10:09.412244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.412197 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:10:09.412348 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.412293 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:10:09.412348 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.412341 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:10:09.573380 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.573335 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" event={"ID":"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2","Type":"ContainerStarted","Data":"8acb40957c7935f762e19fe338a48d628c944109f0edeaf27a389412761a00cb"} Jul 13 07:10:09.573551 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.573395 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" event={"ID":"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2","Type":"ContainerStarted","Data":"5129e0f84a223aec49ac5a529aaedbec6818b1155670ad9b1745d76c07a79748"} Jul 13 07:10:09.592299 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:09.592246 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" podStartSLOduration=1.592230071 podStartE2EDuration="1.592230071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:10:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:10:09.590997181 +0000 UTC m=+462.765501005" watchObservedRunningTime="2026-07-13 07:10:09.592230071 +0000 UTC m=+462.766733875" Jul 13 07:10:10.249543 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:10.249507 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:10.254109 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:10.254081 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:10.578045 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:10.577960 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:10.578987 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:10.578965 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:10:12.303419 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.303387 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:12.311676 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.311650 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-svsjm"] Jul 13 07:10:12.315136 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.315119 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:12.317452 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.317433 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 07:10:12.327170 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.326173 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-svsjm"] Jul 13 07:10:12.411048 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.411016 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-config-file\") pod \"limitador-limitador-64c94fd64f-svsjm\" (UID: \"1fc74037-869b-4aa9-bfa3-b5ec29298ea1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:12.411241 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.411140 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jltxl\" (UniqueName: \"kubernetes.io/projected/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-kube-api-access-jltxl\") pod \"limitador-limitador-64c94fd64f-svsjm\" (UID: \"1fc74037-869b-4aa9-bfa3-b5ec29298ea1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:12.415857 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.415830 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-svsjm"] Jul 13 07:10:12.417566 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.417541 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:12.511774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.511735 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltxl\" (UniqueName: \"kubernetes.io/projected/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-kube-api-access-jltxl\") pod \"limitador-limitador-64c94fd64f-svsjm\" (UID: \"1fc74037-869b-4aa9-bfa3-b5ec29298ea1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:12.511948 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.511818 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-config-file\") pod \"limitador-limitador-64c94fd64f-svsjm\" (UID: \"1fc74037-869b-4aa9-bfa3-b5ec29298ea1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:12.512612 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.512590 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-config-file\") pod \"limitador-limitador-64c94fd64f-svsjm\" (UID: \"1fc74037-869b-4aa9-bfa3-b5ec29298ea1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:12.518299 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.518270 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:12.521107 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.521081 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jltxl\" (UniqueName: \"kubernetes.io/projected/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-kube-api-access-jltxl\") pod \"limitador-limitador-64c94fd64f-svsjm\" (UID: \"1fc74037-869b-4aa9-bfa3-b5ec29298ea1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:12.632195 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.632105 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:12.771703 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:12.771679 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-svsjm"] Jul 13 07:10:12.773999 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:10:12.773969 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc74037_869b_4aa9_bfa3_b5ec29298ea1.slice/crio-c89bf024c0be44f100f255d8217fb4397676142aaf5bf1c7d0b9d6f7f7b0c836 WatchSource:0}: Error finding container c89bf024c0be44f100f255d8217fb4397676142aaf5bf1c7d0b9d6f7f7b0c836: Status 404 returned error can't find the container with id c89bf024c0be44f100f255d8217fb4397676142aaf5bf1c7d0b9d6f7f7b0c836 Jul 13 07:10:13.160671 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:13.160633 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:13.262047 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:13.262016 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:13.593861 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:13.593822 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" event={"ID":"1fc74037-869b-4aa9-bfa3-b5ec29298ea1","Type":"ContainerStarted","Data":"c89bf024c0be44f100f255d8217fb4397676142aaf5bf1c7d0b9d6f7f7b0c836"} Jul 13 07:10:16.612381 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:16.612340 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" event={"ID":"1fc74037-869b-4aa9-bfa3-b5ec29298ea1","Type":"ContainerStarted","Data":"0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0"} Jul 13 07:10:16.612799 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:16.612412 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:16.629445 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:16.629395 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" podStartSLOduration=1.6008234799999999 podStartE2EDuration="4.629378585s" totalImagesPullingTime="3.028555105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:10:16.627269146 +0000 UTC m=+469.801772955" watchObservedRunningTime="2026-07-13 07:10:16.629378585 +0000 UTC m=+469.803882391" Jul 13 07:10:27.618587 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:27.618517 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:29.446931 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:29.446892 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-svsjm"] Jul 13 07:10:29.447402 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:29.447069 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" podUID="1fc74037-869b-4aa9-bfa3-b5ec29298ea1" containerName="limitador" containerID="cri-o://0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0" gracePeriod=30 Jul 13 07:10:29.998865 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:29.998839 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:30.179835 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.179751 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-config-file\") pod \"1fc74037-869b-4aa9-bfa3-b5ec29298ea1\" (UID: \"1fc74037-869b-4aa9-bfa3-b5ec29298ea1\") " Jul 13 07:10:30.179835 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.179782 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jltxl\" (UniqueName: \"kubernetes.io/projected/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-kube-api-access-jltxl\") pod \"1fc74037-869b-4aa9-bfa3-b5ec29298ea1\" (UID: \"1fc74037-869b-4aa9-bfa3-b5ec29298ea1\") " Jul 13 07:10:30.180152 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.180127 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-config-file" (OuterVolumeSpecName: "config-file") pod "1fc74037-869b-4aa9-bfa3-b5ec29298ea1" (UID: "1fc74037-869b-4aa9-bfa3-b5ec29298ea1"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:10:30.181967 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.181945 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-kube-api-access-jltxl" (OuterVolumeSpecName: "kube-api-access-jltxl") pod "1fc74037-869b-4aa9-bfa3-b5ec29298ea1" (UID: "1fc74037-869b-4aa9-bfa3-b5ec29298ea1"). InnerVolumeSpecName "kube-api-access-jltxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:10:30.281400 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.281375 2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-config-file\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:10:30.281400 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.281398 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jltxl\" (UniqueName: \"kubernetes.io/projected/1fc74037-869b-4aa9-bfa3-b5ec29298ea1-kube-api-access-jltxl\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:10:30.686215 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.686183 2583 generic.go:356] "Generic (PLEG): container finished" podID="1fc74037-869b-4aa9-bfa3-b5ec29298ea1" containerID="0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0" exitCode=0 Jul 13 07:10:30.686631 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.686251 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" event={"ID":"1fc74037-869b-4aa9-bfa3-b5ec29298ea1","Type":"ContainerDied","Data":"0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0"} Jul 13 07:10:30.686631 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.686290 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" event={"ID":"1fc74037-869b-4aa9-bfa3-b5ec29298ea1","Type":"ContainerDied","Data":"c89bf024c0be44f100f255d8217fb4397676142aaf5bf1c7d0b9d6f7f7b0c836"} Jul 13 07:10:30.686631 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.686315 2583 scope.go:117] "RemoveContainer" containerID="0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0" Jul 13 07:10:30.686631 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.686252 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-svsjm" Jul 13 07:10:30.695310 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.695291 2583 scope.go:117] "RemoveContainer" containerID="0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0" Jul 13 07:10:30.695586 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:10:30.695558 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0\": container with ID starting with 0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0 not found: ID does not exist" containerID="0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0" Jul 13 07:10:30.695697 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.695590 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0"} err="failed to get container status \"0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0\": rpc error: code = NotFound desc = could not find container \"0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0\": container with ID starting with 0992cb957d9b2647ed751eba5c9cbf67b1ab8aea9241ecf2977ae20ee02587c0 not found: ID does not exist" Jul 13 07:10:30.707580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.707553 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-svsjm"] Jul 13 07:10:30.709273 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:30.709248 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-svsjm"] Jul 13 07:10:31.428964 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:31.428922 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fc74037-869b-4aa9-bfa3-b5ec29298ea1" path="/var/lib/kubelet/pods/1fc74037-869b-4aa9-bfa3-b5ec29298ea1/volumes" Jul 13 07:10:31.841262 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:31.841236 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:31.935769 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:31.935740 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:32.268136 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.268106 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8"] Jul 13 07:10:32.268713 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.268696 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc74037-869b-4aa9-bfa3-b5ec29298ea1" containerName="limitador" Jul 13 07:10:32.268757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.268716 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc74037-869b-4aa9-bfa3-b5ec29298ea1" containerName="limitador" Jul 13 07:10:32.268820 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.268808 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc74037-869b-4aa9-bfa3-b5ec29298ea1" containerName="limitador" Jul 13 07:10:32.271776 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.271757 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" Jul 13 07:10:32.274218 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.274183 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 13 07:10:32.274334 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.274320 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-l7zp6\"" Jul 13 07:10:32.274413 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.274394 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 07:10:32.275585 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.275503 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 07:10:32.278298 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.278266 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8"] Jul 13 07:10:32.297461 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.297443 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:32.400840 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.400816 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2hwd\" (UniqueName: \"kubernetes.io/projected/c9833f34-dae3-4e94-9394-2eb2f84318f1-kube-api-access-t2hwd\") pod \"postgres-69fb4594cf-lrbf8\" (UID: \"c9833f34-dae3-4e94-9394-2eb2f84318f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" Jul 13 07:10:32.400953 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.400859 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9833f34-dae3-4e94-9394-2eb2f84318f1-data\") pod \"postgres-69fb4594cf-lrbf8\" (UID: \"c9833f34-dae3-4e94-9394-2eb2f84318f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" Jul 13 07:10:32.501445 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.501418 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9833f34-dae3-4e94-9394-2eb2f84318f1-data\") pod \"postgres-69fb4594cf-lrbf8\" (UID: \"c9833f34-dae3-4e94-9394-2eb2f84318f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" Jul 13 07:10:32.501548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.501522 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2hwd\" (UniqueName: \"kubernetes.io/projected/c9833f34-dae3-4e94-9394-2eb2f84318f1-kube-api-access-t2hwd\") pod \"postgres-69fb4594cf-lrbf8\" (UID: \"c9833f34-dae3-4e94-9394-2eb2f84318f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" Jul 13 07:10:32.501857 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.501836 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9833f34-dae3-4e94-9394-2eb2f84318f1-data\") pod \"postgres-69fb4594cf-lrbf8\" (UID: \"c9833f34-dae3-4e94-9394-2eb2f84318f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" Jul 13 07:10:32.509527 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.509505 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2hwd\" (UniqueName: \"kubernetes.io/projected/c9833f34-dae3-4e94-9394-2eb2f84318f1-kube-api-access-t2hwd\") pod \"postgres-69fb4594cf-lrbf8\" (UID: \"c9833f34-dae3-4e94-9394-2eb2f84318f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" Jul 13 07:10:32.583183 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.583128 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" Jul 13 07:10:32.728392 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:32.728361 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8"] Jul 13 07:10:32.731532 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:10:32.731503 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9833f34_dae3_4e94_9394_2eb2f84318f1.slice/crio-e590579a4d692db685920c38009970c85764f9fcbb95cb3a90fa3756e41f3c3e WatchSource:0}: Error finding container e590579a4d692db685920c38009970c85764f9fcbb95cb3a90fa3756e41f3c3e: Status 404 returned error can't find the container with id e590579a4d692db685920c38009970c85764f9fcbb95cb3a90fa3756e41f3c3e Jul 13 07:10:33.704291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:33.704251 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" event={"ID":"c9833f34-dae3-4e94-9394-2eb2f84318f1","Type":"ContainerStarted","Data":"e590579a4d692db685920c38009970c85764f9fcbb95cb3a90fa3756e41f3c3e"} Jul 13 07:10:37.729278 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:37.729238 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" event={"ID":"c9833f34-dae3-4e94-9394-2eb2f84318f1","Type":"ContainerStarted","Data":"2debf1b82392b10d6ccf6075508dcd8cc2b4ad9fea20e2f8af209a9a6aa02d03"} Jul 13 07:10:37.729680 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:37.729353 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" Jul 13 07:10:37.748041 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:37.747988 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" podStartSLOduration=1.253265362 podStartE2EDuration="5.747971664s" totalImagesPullingTime="4.494706302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:10:37.74604392 +0000 UTC m=+490.920547748" watchObservedRunningTime="2026-07-13 07:10:37.747971664 +0000 UTC m=+490.922475522" Jul 13 07:10:43.763235 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:43.763189 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrbf8" Jul 13 07:10:44.086102 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.086021 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:44.206732 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.206699 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:44.519710 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.519676 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-s4hws"] Jul 13 07:10:44.526197 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.526171 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s4hws" Jul 13 07:10:44.529956 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.529925 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 13 07:10:44.530101 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.530034 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d5rk4\"" Jul 13 07:10:44.532267 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.532243 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-s4hws"] Jul 13 07:10:44.580482 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.580448 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-s4hws"] Jul 13 07:10:44.580767 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:10:44.580747 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-x6cgq tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-x6cgq tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-s4hws" podUID="13fc8a13-0c16-4f46-85df-30493574a24f" Jul 13 07:10:44.612122 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.612079 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6cgq\" (UniqueName: \"kubernetes.io/projected/13fc8a13-0c16-4f46-85df-30493574a24f-kube-api-access-x6cgq\") pod \"authorino-76877994-s4hws\" (UID: \"13fc8a13-0c16-4f46-85df-30493574a24f\") " pod="kuadrant-system/authorino-76877994-s4hws" Jul 13 07:10:44.612122 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.612126 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/13fc8a13-0c16-4f46-85df-30493574a24f-tls-cert\") pod \"authorino-76877994-s4hws\" (UID: \"13fc8a13-0c16-4f46-85df-30493574a24f\") " pod="kuadrant-system/authorino-76877994-s4hws" Jul 13 07:10:44.713370 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.713336 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6cgq\" (UniqueName: \"kubernetes.io/projected/13fc8a13-0c16-4f46-85df-30493574a24f-kube-api-access-x6cgq\") pod \"authorino-76877994-s4hws\" (UID: \"13fc8a13-0c16-4f46-85df-30493574a24f\") " pod="kuadrant-system/authorino-76877994-s4hws" Jul 13 07:10:44.713518 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.713376 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/13fc8a13-0c16-4f46-85df-30493574a24f-tls-cert\") pod \"authorino-76877994-s4hws\" (UID: \"13fc8a13-0c16-4f46-85df-30493574a24f\") " pod="kuadrant-system/authorino-76877994-s4hws" Jul 13 07:10:44.715757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.715727 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/13fc8a13-0c16-4f46-85df-30493574a24f-tls-cert\") pod \"authorino-76877994-s4hws\" (UID: \"13fc8a13-0c16-4f46-85df-30493574a24f\") " pod="kuadrant-system/authorino-76877994-s4hws" Jul 13 07:10:44.720995 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.720970 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6cgq\" (UniqueName: \"kubernetes.io/projected/13fc8a13-0c16-4f46-85df-30493574a24f-kube-api-access-x6cgq\") pod \"authorino-76877994-s4hws\" (UID: \"13fc8a13-0c16-4f46-85df-30493574a24f\") " pod="kuadrant-system/authorino-76877994-s4hws" Jul 13 07:10:44.760876 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.760846 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s4hws" Jul 13 07:10:44.768534 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.768511 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s4hws" Jul 13 07:10:44.813830 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.813732 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6cgq\" (UniqueName: \"kubernetes.io/projected/13fc8a13-0c16-4f46-85df-30493574a24f-kube-api-access-x6cgq\") pod \"13fc8a13-0c16-4f46-85df-30493574a24f\" (UID: \"13fc8a13-0c16-4f46-85df-30493574a24f\") " Jul 13 07:10:44.813968 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.813831 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/13fc8a13-0c16-4f46-85df-30493574a24f-tls-cert\") pod \"13fc8a13-0c16-4f46-85df-30493574a24f\" (UID: \"13fc8a13-0c16-4f46-85df-30493574a24f\") " Jul 13 07:10:44.815921 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.815886 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13fc8a13-0c16-4f46-85df-30493574a24f-kube-api-access-x6cgq" (OuterVolumeSpecName: "kube-api-access-x6cgq") pod "13fc8a13-0c16-4f46-85df-30493574a24f" (UID: "13fc8a13-0c16-4f46-85df-30493574a24f"). InnerVolumeSpecName "kube-api-access-x6cgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:10:44.816028 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.815952 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13fc8a13-0c16-4f46-85df-30493574a24f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "13fc8a13-0c16-4f46-85df-30493574a24f" (UID: "13fc8a13-0c16-4f46-85df-30493574a24f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:10:44.915091 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.915055 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6cgq\" (UniqueName: \"kubernetes.io/projected/13fc8a13-0c16-4f46-85df-30493574a24f-kube-api-access-x6cgq\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:10:44.915091 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:44.915089 2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/13fc8a13-0c16-4f46-85df-30493574a24f-tls-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:10:45.766068 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:45.766038 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s4hws" Jul 13 07:10:45.791731 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:45.791700 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-s4hws"] Jul 13 07:10:45.795627 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:45.795600 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-s4hws"] Jul 13 07:10:47.428379 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:47.428346 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13fc8a13-0c16-4f46-85df-30493574a24f" path="/var/lib/kubelet/pods/13fc8a13-0c16-4f46-85df-30493574a24f/volumes" Jul 13 07:10:50.851810 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:50.851776 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:50.869906 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:50.869874 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8476d89dc8-n6zhh"] Jul 13 07:10:50.872455 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:50.872434 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:10:50.875357 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:50.875336 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pnj56\"" Jul 13 07:10:50.875476 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:50.875354 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 07:10:50.875476 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:50.875342 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 07:10:50.884375 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:50.884329 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8476d89dc8-n6zhh"] Jul 13 07:10:50.945120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:50.945085 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:50.966474 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:50.966441 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78dfc\" (UniqueName: \"kubernetes.io/projected/697b51b4-85d1-498e-9e44-c0342187db2a-kube-api-access-78dfc\") pod \"maas-controller-8476d89dc8-n6zhh\" (UID: \"697b51b4-85d1-498e-9e44-c0342187db2a\") " pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:10:50.966648 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:50.966520 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/697b51b4-85d1-498e-9e44-c0342187db2a-webhook-cert\") pod \"maas-controller-8476d89dc8-n6zhh\" (UID: \"697b51b4-85d1-498e-9e44-c0342187db2a\") " pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:10:51.030175 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.030138 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:51.034246 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.034193 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b7565664c-ljp2f"] Jul 13 07:10:51.036892 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.036872 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:10:51.047610 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.047581 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7565664c-ljp2f"] Jul 13 07:10:51.067619 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.067580 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78dfc\" (UniqueName: \"kubernetes.io/projected/697b51b4-85d1-498e-9e44-c0342187db2a-kube-api-access-78dfc\") pod \"maas-controller-8476d89dc8-n6zhh\" (UID: \"697b51b4-85d1-498e-9e44-c0342187db2a\") " pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:10:51.067619 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.067624 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c708eb8-2b69-49b4-a3ba-62429908a33b-webhook-cert\") pod \"maas-controller-5b7565664c-ljp2f\" (UID: \"9c708eb8-2b69-49b4-a3ba-62429908a33b\") " pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:10:51.067876 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.067675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/697b51b4-85d1-498e-9e44-c0342187db2a-webhook-cert\") pod \"maas-controller-8476d89dc8-n6zhh\" (UID: \"697b51b4-85d1-498e-9e44-c0342187db2a\") " pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:10:51.067876 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.067712 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbkt9\" (UniqueName: \"kubernetes.io/projected/9c708eb8-2b69-49b4-a3ba-62429908a33b-kube-api-access-vbkt9\") pod \"maas-controller-5b7565664c-ljp2f\" (UID: \"9c708eb8-2b69-49b4-a3ba-62429908a33b\") " pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:10:51.070219 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.070178 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/697b51b4-85d1-498e-9e44-c0342187db2a-webhook-cert\") pod \"maas-controller-8476d89dc8-n6zhh\" (UID: \"697b51b4-85d1-498e-9e44-c0342187db2a\") " pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:10:51.075511 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.075487 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78dfc\" (UniqueName: \"kubernetes.io/projected/697b51b4-85d1-498e-9e44-c0342187db2a-kube-api-access-78dfc\") pod \"maas-controller-8476d89dc8-n6zhh\" (UID: \"697b51b4-85d1-498e-9e44-c0342187db2a\") " pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:10:51.168854 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.168747 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c708eb8-2b69-49b4-a3ba-62429908a33b-webhook-cert\") pod \"maas-controller-5b7565664c-ljp2f\" (UID: \"9c708eb8-2b69-49b4-a3ba-62429908a33b\") " pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:10:51.169034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.168888 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbkt9\" (UniqueName: \"kubernetes.io/projected/9c708eb8-2b69-49b4-a3ba-62429908a33b-kube-api-access-vbkt9\") pod \"maas-controller-5b7565664c-ljp2f\" (UID: \"9c708eb8-2b69-49b4-a3ba-62429908a33b\") " pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:10:51.171165 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.171142 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c708eb8-2b69-49b4-a3ba-62429908a33b-webhook-cert\") pod \"maas-controller-5b7565664c-ljp2f\" (UID: \"9c708eb8-2b69-49b4-a3ba-62429908a33b\") " pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:10:51.177803 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.177777 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbkt9\" (UniqueName: \"kubernetes.io/projected/9c708eb8-2b69-49b4-a3ba-62429908a33b-kube-api-access-vbkt9\") pod \"maas-controller-5b7565664c-ljp2f\" (UID: \"9c708eb8-2b69-49b4-a3ba-62429908a33b\") " pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:10:51.182610 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.182588 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:10:51.317723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.317697 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8476d89dc8-n6zhh"] Jul 13 07:10:51.319782 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:10:51.319746 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod697b51b4_85d1_498e_9e44_c0342187db2a.slice/crio-8a2b514b4f5f555d1ff1afd5b4d3050ec250fcb259a14ea5fd4bb606fcaefded WatchSource:0}: Error finding container 8a2b514b4f5f555d1ff1afd5b4d3050ec250fcb259a14ea5fd4bb606fcaefded: Status 404 returned error can't find the container with id 8a2b514b4f5f555d1ff1afd5b4d3050ec250fcb259a14ea5fd4bb606fcaefded Jul 13 07:10:51.347140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.347104 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:10:51.481049 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.480971 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7565664c-ljp2f"] Jul 13 07:10:51.483176 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:10:51.483152 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c708eb8_2b69_49b4_a3ba_62429908a33b.slice/crio-6a679de1980c436f6cbb4c0135da1a971d481465b06412bb068b7cd18e8647a5 WatchSource:0}: Error finding container 6a679de1980c436f6cbb4c0135da1a971d481465b06412bb068b7cd18e8647a5: Status 404 returned error can't find the container with id 6a679de1980c436f6cbb4c0135da1a971d481465b06412bb068b7cd18e8647a5 Jul 13 07:10:51.794973 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.794937 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" event={"ID":"697b51b4-85d1-498e-9e44-c0342187db2a","Type":"ContainerStarted","Data":"8a2b514b4f5f555d1ff1afd5b4d3050ec250fcb259a14ea5fd4bb606fcaefded"} Jul 13 07:10:51.796013 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:51.795989 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7565664c-ljp2f" event={"ID":"9c708eb8-2b69-49b4-a3ba-62429908a33b","Type":"ContainerStarted","Data":"6a679de1980c436f6cbb4c0135da1a971d481465b06412bb068b7cd18e8647a5"} Jul 13 07:10:53.666749 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:53.666707 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:53.810405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:53.810308 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" event={"ID":"697b51b4-85d1-498e-9e44-c0342187db2a","Type":"ContainerStarted","Data":"583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505"} Jul 13 07:10:53.810405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:53.810386 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:10:53.811779 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:53.811748 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7565664c-ljp2f" event={"ID":"9c708eb8-2b69-49b4-a3ba-62429908a33b","Type":"ContainerStarted","Data":"511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f"} Jul 13 07:10:53.811914 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:53.811880 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:10:53.842733 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:53.842697 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:10:53.844973 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:53.844921 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" podStartSLOduration=1.671998022 podStartE2EDuration="3.844903233s" totalImagesPullingTime="2.172905211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:10:53.843679716 +0000 UTC m=+507.018183542" watchObservedRunningTime="2026-07-13 07:10:53.844903233 +0000 UTC m=+507.019407041" Jul 13 07:10:53.880419 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:10:53.880376 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b7565664c-ljp2f" podStartSLOduration=0.880406854 podStartE2EDuration="2.880358398s" totalImagesPullingTime="1.999951544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:10:53.87394756 +0000 UTC m=+507.048451366" watchObservedRunningTime="2026-07-13 07:10:53.880358398 +0000 UTC m=+507.054862209" Jul 13 07:11:04.821652 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:04.821619 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:11:04.822161 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:04.821910 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:11:04.878182 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:04.878150 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8476d89dc8-n6zhh"] Jul 13 07:11:04.878394 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:04.878347 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" podUID="697b51b4-85d1-498e-9e44-c0342187db2a" containerName="manager" containerID="cri-o://583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505" gracePeriod=30 Jul 13 07:11:05.116226 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.116190 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:11:05.199829 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.199799 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/697b51b4-85d1-498e-9e44-c0342187db2a-webhook-cert\") pod \"697b51b4-85d1-498e-9e44-c0342187db2a\" (UID: \"697b51b4-85d1-498e-9e44-c0342187db2a\") " Jul 13 07:11:05.199944 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.199905 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78dfc\" (UniqueName: \"kubernetes.io/projected/697b51b4-85d1-498e-9e44-c0342187db2a-kube-api-access-78dfc\") pod \"697b51b4-85d1-498e-9e44-c0342187db2a\" (UID: \"697b51b4-85d1-498e-9e44-c0342187db2a\") " Jul 13 07:11:05.201970 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.201942 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/697b51b4-85d1-498e-9e44-c0342187db2a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "697b51b4-85d1-498e-9e44-c0342187db2a" (UID: "697b51b4-85d1-498e-9e44-c0342187db2a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:11:05.202069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.201951 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/697b51b4-85d1-498e-9e44-c0342187db2a-kube-api-access-78dfc" (OuterVolumeSpecName: "kube-api-access-78dfc") pod "697b51b4-85d1-498e-9e44-c0342187db2a" (UID: "697b51b4-85d1-498e-9e44-c0342187db2a"). InnerVolumeSpecName "kube-api-access-78dfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:11:05.300714 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.300693 2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/697b51b4-85d1-498e-9e44-c0342187db2a-webhook-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:11:05.300714 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.300713 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78dfc\" (UniqueName: \"kubernetes.io/projected/697b51b4-85d1-498e-9e44-c0342187db2a-kube-api-access-78dfc\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:11:05.875497 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.875459 2583 generic.go:356] "Generic (PLEG): container finished" podID="697b51b4-85d1-498e-9e44-c0342187db2a" containerID="583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505" exitCode=0 Jul 13 07:11:05.875921 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.875502 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" Jul 13 07:11:05.875921 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.875546 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" event={"ID":"697b51b4-85d1-498e-9e44-c0342187db2a","Type":"ContainerDied","Data":"583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505"} Jul 13 07:11:05.875921 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.875602 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8476d89dc8-n6zhh" event={"ID":"697b51b4-85d1-498e-9e44-c0342187db2a","Type":"ContainerDied","Data":"8a2b514b4f5f555d1ff1afd5b4d3050ec250fcb259a14ea5fd4bb606fcaefded"} Jul 13 07:11:05.875921 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.875631 2583 scope.go:117] "RemoveContainer" containerID="583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505" Jul 13 07:11:05.885383 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.885213 2583 scope.go:117] "RemoveContainer" containerID="583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505" Jul 13 07:11:05.885514 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:11:05.885494 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505\": container with ID starting with 583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505 not found: ID does not exist" containerID="583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505" Jul 13 07:11:05.885555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.885523 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505"} err="failed to get container status \"583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505\": rpc error: code = NotFound desc = could not find container \"583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505\": container with ID starting with 583cad4cb4906980ba467677ab75b80748add13b2f1cb57d0833899301cca505 not found: ID does not exist" Jul 13 07:11:05.893369 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.893344 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8476d89dc8-n6zhh"] Jul 13 07:11:05.898033 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:05.898008 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8476d89dc8-n6zhh"] Jul 13 07:11:07.428608 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:07.428563 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="697b51b4-85d1-498e-9e44-c0342187db2a" path="/var/lib/kubelet/pods/697b51b4-85d1-498e-9e44-c0342187db2a/volumes" Jul 13 07:11:24.497971 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:24.497930 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:11:24.882642 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:24.882557 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:11:24.932453 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:24.932420 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x"] Jul 13 07:11:24.932838 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:24.932813 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="697b51b4-85d1-498e-9e44-c0342187db2a" containerName="manager" Jul 13 07:11:24.932883 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:24.932840 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="697b51b4-85d1-498e-9e44-c0342187db2a" containerName="manager" Jul 13 07:11:24.932935 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:24.932925 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="697b51b4-85d1-498e-9e44-c0342187db2a" containerName="manager" Jul 13 07:11:24.935026 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:24.935009 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:24.937996 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:24.937969 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8rh7l\"" Jul 13 07:11:24.937996 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:24.937978 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 13 07:11:24.954064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:24.954040 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x"] Jul 13 07:11:25.063313 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.063279 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a6009b-b9df-418e-aa00-1e3b43c03d76-maas-api-tls\") pod \"maas-api-b9ccf5cf-z8c8x\" (UID: \"69a6009b-b9df-418e-aa00-1e3b43c03d76\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:25.063470 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.063325 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh8gb\" (UniqueName: \"kubernetes.io/projected/69a6009b-b9df-418e-aa00-1e3b43c03d76-kube-api-access-rh8gb\") pod \"maas-api-b9ccf5cf-z8c8x\" (UID: \"69a6009b-b9df-418e-aa00-1e3b43c03d76\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:25.063470 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.063403 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a6009b-b9df-418e-aa00-1e3b43c03d76-openshift-service-ca\") pod \"maas-api-b9ccf5cf-z8c8x\" (UID: \"69a6009b-b9df-418e-aa00-1e3b43c03d76\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:25.137250 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.137155 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:11:25.164483 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.164450 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a6009b-b9df-418e-aa00-1e3b43c03d76-openshift-service-ca\") pod \"maas-api-b9ccf5cf-z8c8x\" (UID: \"69a6009b-b9df-418e-aa00-1e3b43c03d76\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:25.164620 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.164560 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a6009b-b9df-418e-aa00-1e3b43c03d76-maas-api-tls\") pod \"maas-api-b9ccf5cf-z8c8x\" (UID: \"69a6009b-b9df-418e-aa00-1e3b43c03d76\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:25.164620 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.164603 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh8gb\" (UniqueName: \"kubernetes.io/projected/69a6009b-b9df-418e-aa00-1e3b43c03d76-kube-api-access-rh8gb\") pod \"maas-api-b9ccf5cf-z8c8x\" (UID: \"69a6009b-b9df-418e-aa00-1e3b43c03d76\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:25.165022 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.164993 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a6009b-b9df-418e-aa00-1e3b43c03d76-openshift-service-ca\") pod \"maas-api-b9ccf5cf-z8c8x\" (UID: \"69a6009b-b9df-418e-aa00-1e3b43c03d76\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:25.166924 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.166905 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a6009b-b9df-418e-aa00-1e3b43c03d76-maas-api-tls\") pod \"maas-api-b9ccf5cf-z8c8x\" (UID: \"69a6009b-b9df-418e-aa00-1e3b43c03d76\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:25.173046 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.173022 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh8gb\" (UniqueName: \"kubernetes.io/projected/69a6009b-b9df-418e-aa00-1e3b43c03d76-kube-api-access-rh8gb\") pod \"maas-api-b9ccf5cf-z8c8x\" (UID: \"69a6009b-b9df-418e-aa00-1e3b43c03d76\") " pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:25.245564 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.245540 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:25.357905 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.357874 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:11:25.401321 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.401286 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x"] Jul 13 07:11:25.403979 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:11:25.403952 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a6009b_b9df_418e_aa00_1e3b43c03d76.slice/crio-8d9ab3ac7526f50cdd33ac32f877af435623fffd9fd7df0acb97ed91a4262570 WatchSource:0}: Error finding container 8d9ab3ac7526f50cdd33ac32f877af435623fffd9fd7df0acb97ed91a4262570: Status 404 returned error can't find the container with id 8d9ab3ac7526f50cdd33ac32f877af435623fffd9fd7df0acb97ed91a4262570 Jul 13 07:11:25.463587 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.463555 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:11:25.492640 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.492614 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-bb8j4"] Jul 13 07:11:25.495702 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.495684 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" Jul 13 07:11:25.500746 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.500729 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tfrcn\"" Jul 13 07:11:25.501084 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.500956 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 13 07:11:25.508728 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.508707 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-bb8j4"] Jul 13 07:11:25.641526 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.641504 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-thlj2"] Jul 13 07:11:25.649805 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.649779 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-thlj2" Jul 13 07:11:25.654864 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.654796 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-thlj2"] Jul 13 07:11:25.668676 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.668648 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/540a0df8-8613-41c9-8218-e0c1fbaa2e4a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-bb8j4\" (UID: \"540a0df8-8613-41c9-8218-e0c1fbaa2e4a\") " pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" Jul 13 07:11:25.668810 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.668785 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvztc\" (UniqueName: \"kubernetes.io/projected/540a0df8-8613-41c9-8218-e0c1fbaa2e4a-kube-api-access-cvztc\") pod \"payload-pre-processing-7964457696-bb8j4\" (UID: \"540a0df8-8613-41c9-8218-e0c1fbaa2e4a\") " pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" Jul 13 07:11:25.770403 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.770368 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl4pw\" (UniqueName: \"kubernetes.io/projected/68e30d4a-f566-4cfe-a2c5-4f61be32aead-kube-api-access-pl4pw\") pod \"payload-processing-58f86c7798-thlj2\" (UID: \"68e30d4a-f566-4cfe-a2c5-4f61be32aead\") " pod="openshift-ingress/payload-processing-58f86c7798-thlj2" Jul 13 07:11:25.770580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.770470 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/540a0df8-8613-41c9-8218-e0c1fbaa2e4a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-bb8j4\" (UID: \"540a0df8-8613-41c9-8218-e0c1fbaa2e4a\") " pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" Jul 13 07:11:25.770580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.770536 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e30d4a-f566-4cfe-a2c5-4f61be32aead-plugins-config-volume\") pod \"payload-processing-58f86c7798-thlj2\" (UID: \"68e30d4a-f566-4cfe-a2c5-4f61be32aead\") " pod="openshift-ingress/payload-processing-58f86c7798-thlj2" Jul 13 07:11:25.770580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.770558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvztc\" (UniqueName: \"kubernetes.io/projected/540a0df8-8613-41c9-8218-e0c1fbaa2e4a-kube-api-access-cvztc\") pod \"payload-pre-processing-7964457696-bb8j4\" (UID: \"540a0df8-8613-41c9-8218-e0c1fbaa2e4a\") " pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" Jul 13 07:11:25.771173 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.771147 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/540a0df8-8613-41c9-8218-e0c1fbaa2e4a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-bb8j4\" (UID: \"540a0df8-8613-41c9-8218-e0c1fbaa2e4a\") " pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" Jul 13 07:11:25.779796 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.779773 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvztc\" (UniqueName: \"kubernetes.io/projected/540a0df8-8613-41c9-8218-e0c1fbaa2e4a-kube-api-access-cvztc\") pod \"payload-pre-processing-7964457696-bb8j4\" (UID: \"540a0df8-8613-41c9-8218-e0c1fbaa2e4a\") " pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" Jul 13 07:11:25.805791 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.805764 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" Jul 13 07:11:25.871221 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.871179 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e30d4a-f566-4cfe-a2c5-4f61be32aead-plugins-config-volume\") pod \"payload-processing-58f86c7798-thlj2\" (UID: \"68e30d4a-f566-4cfe-a2c5-4f61be32aead\") " pod="openshift-ingress/payload-processing-58f86c7798-thlj2" Jul 13 07:11:25.871353 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.871315 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl4pw\" (UniqueName: \"kubernetes.io/projected/68e30d4a-f566-4cfe-a2c5-4f61be32aead-kube-api-access-pl4pw\") pod \"payload-processing-58f86c7798-thlj2\" (UID: \"68e30d4a-f566-4cfe-a2c5-4f61be32aead\") " pod="openshift-ingress/payload-processing-58f86c7798-thlj2" Jul 13 07:11:25.871909 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.871888 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e30d4a-f566-4cfe-a2c5-4f61be32aead-plugins-config-volume\") pod \"payload-processing-58f86c7798-thlj2\" (UID: \"68e30d4a-f566-4cfe-a2c5-4f61be32aead\") " pod="openshift-ingress/payload-processing-58f86c7798-thlj2" Jul 13 07:11:25.879805 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.879781 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl4pw\" (UniqueName: \"kubernetes.io/projected/68e30d4a-f566-4cfe-a2c5-4f61be32aead-kube-api-access-pl4pw\") pod \"payload-processing-58f86c7798-thlj2\" (UID: \"68e30d4a-f566-4cfe-a2c5-4f61be32aead\") " pod="openshift-ingress/payload-processing-58f86c7798-thlj2" Jul 13 07:11:25.937877 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.937825 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-bb8j4"] Jul 13 07:11:25.939954 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:11:25.939927 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod540a0df8_8613_41c9_8218_e0c1fbaa2e4a.slice/crio-472a281ce20a4616eeca0117ec56fbfd1fe6c9946b293f6be2b8cf3031bfa1ae WatchSource:0}: Error finding container 472a281ce20a4616eeca0117ec56fbfd1fe6c9946b293f6be2b8cf3031bfa1ae: Status 404 returned error can't find the container with id 472a281ce20a4616eeca0117ec56fbfd1fe6c9946b293f6be2b8cf3031bfa1ae Jul 13 07:11:25.961507 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.961486 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-thlj2" Jul 13 07:11:25.980858 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.980828 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" event={"ID":"540a0df8-8613-41c9-8218-e0c1fbaa2e4a","Type":"ContainerStarted","Data":"472a281ce20a4616eeca0117ec56fbfd1fe6c9946b293f6be2b8cf3031bfa1ae"} Jul 13 07:11:25.985043 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:25.985009 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" event={"ID":"69a6009b-b9df-418e-aa00-1e3b43c03d76","Type":"ContainerStarted","Data":"8d9ab3ac7526f50cdd33ac32f877af435623fffd9fd7df0acb97ed91a4262570"} Jul 13 07:11:26.091885 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:26.091858 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-thlj2"] Jul 13 07:11:26.094584 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:11:26.094556 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e30d4a_f566_4cfe_a2c5_4f61be32aead.slice/crio-39785d0572788d21854a201329497a286623f9d49123f534ec5878bcabc8e525 WatchSource:0}: Error finding container 39785d0572788d21854a201329497a286623f9d49123f534ec5878bcabc8e525: Status 404 returned error can't find the container with id 39785d0572788d21854a201329497a286623f9d49123f534ec5878bcabc8e525 Jul 13 07:11:27.010773 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:27.010696 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-thlj2" event={"ID":"68e30d4a-f566-4cfe-a2c5-4f61be32aead","Type":"ContainerStarted","Data":"39785d0572788d21854a201329497a286623f9d49123f534ec5878bcabc8e525"} Jul 13 07:11:28.020239 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:28.020175 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-thlj2" event={"ID":"68e30d4a-f566-4cfe-a2c5-4f61be32aead","Type":"ContainerStarted","Data":"d766e1b41e8fc8798d9ab214401a5748443ce8aeed7b7e636daee30916887931"} Jul 13 07:11:28.020764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:28.020721 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-thlj2" Jul 13 07:11:28.022345 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:28.022287 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" event={"ID":"540a0df8-8613-41c9-8218-e0c1fbaa2e4a","Type":"ContainerStarted","Data":"9bdad3c9789bed6b1732df91460f52e09e7618fe4cf8d2ae44c8e80222e709aa"} Jul 13 07:11:28.022452 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:28.022427 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" Jul 13 07:11:28.041845 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:28.041796 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-thlj2" podStartSLOduration=1.525276944 podStartE2EDuration="3.041780818s" totalImagesPullingTime="1.516503874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:11:28.039796495 +0000 UTC m=+541.214300304" watchObservedRunningTime="2026-07-13 07:11:28.041780818 +0000 UTC m=+541.216284624" Jul 13 07:11:28.058943 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:28.058895 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" podStartSLOduration=1.385061613 podStartE2EDuration="3.058878827s" totalImagesPullingTime="1.673817214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:11:28.057172104 +0000 UTC m=+541.231675911" watchObservedRunningTime="2026-07-13 07:11:28.058878827 +0000 UTC m=+541.233382633" Jul 13 07:11:29.028699 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:29.028651 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" event={"ID":"69a6009b-b9df-418e-aa00-1e3b43c03d76","Type":"ContainerStarted","Data":"c1c62c0b8d78ff0eaa6603f743c76019186a66fefd25317575b328c35e6d14e5"} Jul 13 07:11:29.029168 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:29.028902 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:29.057474 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:29.057424 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" podStartSLOduration=2.202531808 podStartE2EDuration="5.057409603s" totalImagesPullingTime="2.854877795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:11:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:11:29.056987514 +0000 UTC m=+542.231491373" watchObservedRunningTime="2026-07-13 07:11:29.057409603 +0000 UTC m=+542.231913414" Jul 13 07:11:34.055035 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:34.055004 2583 generic.go:356] "Generic (PLEG): container finished" podID="c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2" containerID="8acb40957c7935f762e19fe338a48d628c944109f0edeaf27a389412761a00cb" exitCode=0 Jul 13 07:11:34.055458 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:34.055080 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" event={"ID":"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2","Type":"ContainerDied","Data":"8acb40957c7935f762e19fe338a48d628c944109f0edeaf27a389412761a00cb"} Jul 13 07:11:34.055458 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:34.055428 2583 scope.go:117] "RemoveContainer" containerID="8acb40957c7935f762e19fe338a48d628c944109f0edeaf27a389412761a00cb" Jul 13 07:11:34.056344 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:34.056321 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:11:34.056411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:34.056382 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:11:34.056447 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:34.056422 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:11:35.038824 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:35.038799 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-b9ccf5cf-z8c8x" Jul 13 07:11:35.063420 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:35.063384 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" event={"ID":"c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2","Type":"ContainerStarted","Data":"c7e1dea5bffc0a7d8b39f8d286ad1c4275944abd322d526006da2647c7543060"} Jul 13 07:11:35.249672 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:35.249637 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:11:35.249840 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:35.249797 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ppflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 13 07:11:35.249902 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:35.249841 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" podUID="c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 13 07:11:36.075165 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:36.075126 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:11:36.169003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:36.168973 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:11:36.249831 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:36.249799 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ppflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 13 07:11:36.249993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:36.249857 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" podUID="c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 13 07:11:37.249593 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:37.249561 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ppflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 13 07:11:37.249968 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:37.249624 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" podUID="c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 13 07:11:38.249804 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:38.249769 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ppflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 13 07:11:38.250227 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:38.249831 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" podUID="c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 13 07:11:39.030537 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:39.030504 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-bb8j4" Jul 13 07:11:39.250510 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:39.250475 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ppflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 13 07:11:39.250873 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:39.250530 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" podUID="c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 13 07:11:40.035607 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:40.035576 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-thlj2" Jul 13 07:11:40.253595 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:40.253560 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:11:40.253951 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:40.253806 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:11:40.254530 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:40.254514 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ppflf" Jul 13 07:11:49.068167 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.068132 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb"] Jul 13 07:11:49.070732 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.070708 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.075415 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.075396 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 07:11:49.075515 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.075440 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2zk6k\"" Jul 13 07:11:49.075906 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.075887 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 07:11:49.076001 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.075964 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 13 07:11:49.086165 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.086134 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb"] Jul 13 07:11:49.114199 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.114173 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:11:49.206279 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.206255 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.206445 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.206295 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.206445 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.206323 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svxcj\" (UniqueName: \"kubernetes.io/projected/1645ff78-f8ee-4259-8921-708b5c7fd45b-kube-api-access-svxcj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.206563 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.206448 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1645ff78-f8ee-4259-8921-708b5c7fd45b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.206563 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.206483 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.206563 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.206549 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.307241 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.307195 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.307396 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.307253 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.307396 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.307312 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.307396 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.307338 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.307396 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.307370 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svxcj\" (UniqueName: \"kubernetes.io/projected/1645ff78-f8ee-4259-8921-708b5c7fd45b-kube-api-access-svxcj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.307616 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.307427 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1645ff78-f8ee-4259-8921-708b5c7fd45b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.307754 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.307732 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.307845 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.307827 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.307888 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.307858 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.309521 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.309498 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1645ff78-f8ee-4259-8921-708b5c7fd45b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.309640 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.309627 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1645ff78-f8ee-4259-8921-708b5c7fd45b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.316853 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.316829 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svxcj\" (UniqueName: \"kubernetes.io/projected/1645ff78-f8ee-4259-8921-708b5c7fd45b-kube-api-access-svxcj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb\" (UID: \"1645ff78-f8ee-4259-8921-708b5c7fd45b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.381088 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.381039 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:49.517513 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:49.517488 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb"] Jul 13 07:11:49.519851 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:11:49.519830 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1645ff78_f8ee_4259_8921_708b5c7fd45b.slice/crio-2128d3d74edebbac778603c5ba62cd53e1b93197489a987544e2fa6fd605b493 WatchSource:0}: Error finding container 2128d3d74edebbac778603c5ba62cd53e1b93197489a987544e2fa6fd605b493: Status 404 returned error can't find the container with id 2128d3d74edebbac778603c5ba62cd53e1b93197489a987544e2fa6fd605b493 Jul 13 07:11:50.139095 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:50.139060 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" event={"ID":"1645ff78-f8ee-4259-8921-708b5c7fd45b","Type":"ContainerStarted","Data":"2128d3d74edebbac778603c5ba62cd53e1b93197489a987544e2fa6fd605b493"} Jul 13 07:11:51.146100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:51.146062 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" event={"ID":"1645ff78-f8ee-4259-8921-708b5c7fd45b","Type":"ContainerStarted","Data":"2ceeec86a7282d9dc7bc9716b35069aa23b230eb417e3c315921779011aa26b7"} Jul 13 07:11:51.178255 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:51.178181 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" podStartSLOduration=1.1275804520000001 podStartE2EDuration="2.178165941s" totalImagesPullingTime="1.050585489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:11:51.177746113 +0000 UTC m=+564.352249920" watchObservedRunningTime="2026-07-13 07:11:51.178165941 +0000 UTC m=+564.352669747" Jul 13 07:11:59.381934 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:59.381892 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:59.382484 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:59.381992 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:11:59.394354 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:11:59.394330 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:12:00.098991 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:00.098946 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:12:00.202090 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:00.202057 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb" Jul 13 07:12:04.107605 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:04.107573 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:12:04.207426 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:04.207389 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:12:07.551821 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.551790 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9"] Jul 13 07:12:07.555197 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.555176 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.559333 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.559316 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 13 07:12:07.583125 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.583100 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9"] Jul 13 07:12:07.608066 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.608041 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:12:07.661596 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.661574 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kr4q\" (UniqueName: \"kubernetes.io/projected/52398baf-ed13-4e45-9968-5067bb1046f6-kube-api-access-8kr4q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.661695 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.661632 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.661738 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.661702 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52398baf-ed13-4e45-9968-5067bb1046f6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.661738 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.661728 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.661807 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.661787 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.661841 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.661809 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.762631 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.762603 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.762746 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.762633 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.762746 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.762658 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kr4q\" (UniqueName: \"kubernetes.io/projected/52398baf-ed13-4e45-9968-5067bb1046f6-kube-api-access-8kr4q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.762746 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.762731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.762946 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.762922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52398baf-ed13-4e45-9968-5067bb1046f6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.762996 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.762989 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.763049 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.763018 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.763099 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.763047 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.763158 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.763126 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.765013 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.764992 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52398baf-ed13-4e45-9968-5067bb1046f6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.765375 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.765353 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52398baf-ed13-4e45-9968-5067bb1046f6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.775340 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.775321 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kr4q\" (UniqueName: \"kubernetes.io/projected/52398baf-ed13-4e45-9968-5067bb1046f6-kube-api-access-8kr4q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7xtc9\" (UID: \"52398baf-ed13-4e45-9968-5067bb1046f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:07.865370 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:07.865318 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:08.000233 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:08.000188 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9"] Jul 13 07:12:08.003447 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:12:08.003416 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52398baf_ed13_4e45_9968_5067bb1046f6.slice/crio-b246fb5d2ab3f8d0f27396da2be69e9fe80ea3b5c6b09f14d9bc7ea3487bf635 WatchSource:0}: Error finding container b246fb5d2ab3f8d0f27396da2be69e9fe80ea3b5c6b09f14d9bc7ea3487bf635: Status 404 returned error can't find the container with id b246fb5d2ab3f8d0f27396da2be69e9fe80ea3b5c6b09f14d9bc7ea3487bf635 Jul 13 07:12:08.232652 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:08.232616 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" event={"ID":"52398baf-ed13-4e45-9968-5067bb1046f6","Type":"ContainerStarted","Data":"4d3467ca829b8540b38b957a6e03721d15adfe80a8fe3d47290ef2e270e8123d"} Jul 13 07:12:08.232753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:08.232684 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" event={"ID":"52398baf-ed13-4e45-9968-5067bb1046f6","Type":"ContainerStarted","Data":"b246fb5d2ab3f8d0f27396da2be69e9fe80ea3b5c6b09f14d9bc7ea3487bf635"} Jul 13 07:12:08.253609 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:08.253568 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" podStartSLOduration=1.095261619 podStartE2EDuration="1.253552341s" totalImagesPullingTime="158.290722ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:12:08.252542382 +0000 UTC m=+581.427046198" watchObservedRunningTime="2026-07-13 07:12:08.253552341 +0000 UTC m=+581.428056147" Jul 13 07:12:09.036082 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:09.036043 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:12:13.339865 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.339832 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j"] Jul 13 07:12:13.342709 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.342688 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.345325 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.345305 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 13 07:12:13.352580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.352551 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j"] Jul 13 07:12:13.440956 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.440921 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:12:13.509117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.509090 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.509258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.509123 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f756c19-d5df-49e5-95f0-f0b7104b5381-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.509258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.509154 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.509258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.509234 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.509372 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.509298 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.509372 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.509332 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jj59\" (UniqueName: \"kubernetes.io/projected/4f756c19-d5df-49e5-95f0-f0b7104b5381-kube-api-access-4jj59\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.610588 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.610523 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.610588 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.610556 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f756c19-d5df-49e5-95f0-f0b7104b5381-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.610588 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.610585 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.610761 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.610630 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.610761 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.610701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.610761 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.610741 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jj59\" (UniqueName: \"kubernetes.io/projected/4f756c19-d5df-49e5-95f0-f0b7104b5381-kube-api-access-4jj59\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.610999 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.610977 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.611069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.611049 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.611115 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.611072 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.612872 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.612856 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f756c19-d5df-49e5-95f0-f0b7104b5381-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.613036 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.613018 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f756c19-d5df-49e5-95f0-f0b7104b5381-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.619734 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.619714 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jj59\" (UniqueName: \"kubernetes.io/projected/4f756c19-d5df-49e5-95f0-f0b7104b5381-kube-api-access-4jj59\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j\" (UID: \"4f756c19-d5df-49e5-95f0-f0b7104b5381\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.654470 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.654442 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:13.802633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:13.800907 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j"] Jul 13 07:12:14.263738 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:14.263661 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" event={"ID":"4f756c19-d5df-49e5-95f0-f0b7104b5381","Type":"ContainerStarted","Data":"eae1ef163b1735532c2b031053296ac3c9e744491795aeb1f7192e0ed645226d"} Jul 13 07:12:14.263738 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:14.263712 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" event={"ID":"4f756c19-d5df-49e5-95f0-f0b7104b5381","Type":"ContainerStarted","Data":"e38b31a642741a12e7e058636b350ccd89f2ea3109875aeef7bd7fc3d508042c"} Jul 13 07:12:14.282753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:14.282709 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" podStartSLOduration=1.12062934 podStartE2EDuration="1.282695218s" totalImagesPullingTime="162.065878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:12:14.281805731 +0000 UTC m=+587.456309539" watchObservedRunningTime="2026-07-13 07:12:14.282695218 +0000 UTC m=+587.457199024" Jul 13 07:12:17.865710 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:17.865677 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:17.866089 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:17.865740 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:17.877604 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:17.877579 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:18.294407 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:18.294381 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7xtc9" Jul 13 07:12:23.655017 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:23.654980 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:23.655493 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:23.655054 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:23.667363 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:23.667330 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:24.326519 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:24.326491 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j" Jul 13 07:12:28.498648 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:28.498614 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:12:53.786932 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:12:53.786860 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:13:05.241634 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:13:05.241601 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:13:32.774764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:13:32.774729 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:13:43.892258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:13:43.892188 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:13:43.997445 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:13:43.997418 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:13:45.082827 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:13:45.082793 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:13:45.207876 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:13:45.207842 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:14:43.715956 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:14:43.715921 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:15:16.381906 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:16.381862 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:15:16.492091 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:16.492064 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b7565664c-ljp2f"] Jul 13 07:15:16.492263 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:16.492242 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b7565664c-ljp2f" podUID="9c708eb8-2b69-49b4-a3ba-62429908a33b" containerName="manager" containerID="cri-o://511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f" gracePeriod=30 Jul 13 07:15:16.740438 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:16.740413 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:15:16.810036 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:16.810011 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbkt9\" (UniqueName: \"kubernetes.io/projected/9c708eb8-2b69-49b4-a3ba-62429908a33b-kube-api-access-vbkt9\") pod \"9c708eb8-2b69-49b4-a3ba-62429908a33b\" (UID: \"9c708eb8-2b69-49b4-a3ba-62429908a33b\") " Jul 13 07:15:16.810166 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:16.810102 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c708eb8-2b69-49b4-a3ba-62429908a33b-webhook-cert\") pod \"9c708eb8-2b69-49b4-a3ba-62429908a33b\" (UID: \"9c708eb8-2b69-49b4-a3ba-62429908a33b\") " Jul 13 07:15:16.811957 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:16.811927 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c708eb8-2b69-49b4-a3ba-62429908a33b-kube-api-access-vbkt9" (OuterVolumeSpecName: "kube-api-access-vbkt9") pod "9c708eb8-2b69-49b4-a3ba-62429908a33b" (UID: "9c708eb8-2b69-49b4-a3ba-62429908a33b"). InnerVolumeSpecName "kube-api-access-vbkt9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:15:16.812082 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:16.811969 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c708eb8-2b69-49b4-a3ba-62429908a33b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9c708eb8-2b69-49b4-a3ba-62429908a33b" (UID: "9c708eb8-2b69-49b4-a3ba-62429908a33b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:15:16.911262 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:16.911195 2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c708eb8-2b69-49b4-a3ba-62429908a33b-webhook-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:15:16.911262 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:16.911229 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbkt9\" (UniqueName: \"kubernetes.io/projected/9c708eb8-2b69-49b4-a3ba-62429908a33b-kube-api-access-vbkt9\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:15:17.191684 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.191594 2583 generic.go:356] "Generic (PLEG): container finished" podID="9c708eb8-2b69-49b4-a3ba-62429908a33b" containerID="511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f" exitCode=0 Jul 13 07:15:17.191684 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.191656 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7565664c-ljp2f" Jul 13 07:15:17.191883 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.191679 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7565664c-ljp2f" event={"ID":"9c708eb8-2b69-49b4-a3ba-62429908a33b","Type":"ContainerDied","Data":"511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f"} Jul 13 07:15:17.191883 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.191757 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7565664c-ljp2f" event={"ID":"9c708eb8-2b69-49b4-a3ba-62429908a33b","Type":"ContainerDied","Data":"6a679de1980c436f6cbb4c0135da1a971d481465b06412bb068b7cd18e8647a5"} Jul 13 07:15:17.191883 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.191780 2583 scope.go:117] "RemoveContainer" containerID="511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f" Jul 13 07:15:17.200717 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.200703 2583 scope.go:117] "RemoveContainer" containerID="511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f" Jul 13 07:15:17.200948 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:15:17.200927 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f\": container with ID starting with 511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f not found: ID does not exist" containerID="511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f" Jul 13 07:15:17.201012 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.200959 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f"} err="failed to get container status \"511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f\": rpc error: code = NotFound desc = could not find container \"511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f\": container with ID starting with 511746e675f272e4f9e1781c313ea408fcf31db96034e91ba9359a27aebadb1f not found: ID does not exist" Jul 13 07:15:17.212371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.212347 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b7565664c-ljp2f"] Jul 13 07:15:17.216726 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.216706 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b7565664c-ljp2f"] Jul 13 07:15:17.428557 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.428520 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c708eb8-2b69-49b4-a3ba-62429908a33b" path="/var/lib/kubelet/pods/9c708eb8-2b69-49b4-a3ba-62429908a33b/volumes" Jul 13 07:15:17.840778 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.840745 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:15:17.958411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.958384 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b7565664c-cprnk"] Jul 13 07:15:17.958978 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.958957 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c708eb8-2b69-49b4-a3ba-62429908a33b" containerName="manager" Jul 13 07:15:17.958978 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.958976 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c708eb8-2b69-49b4-a3ba-62429908a33b" containerName="manager" Jul 13 07:15:17.959103 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.959041 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c708eb8-2b69-49b4-a3ba-62429908a33b" containerName="manager" Jul 13 07:15:17.963774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.963758 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7565664c-cprnk" Jul 13 07:15:17.966441 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.966422 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 07:15:17.967936 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.967913 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 07:15:17.968044 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.967939 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pnj56\"" Jul 13 07:15:17.970494 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:17.970470 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7565664c-cprnk"] Jul 13 07:15:18.120704 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:18.120644 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkbzr\" (UniqueName: \"kubernetes.io/projected/720b787b-8a37-4ce5-beb5-433d5c81961f-kube-api-access-pkbzr\") pod \"maas-controller-5b7565664c-cprnk\" (UID: \"720b787b-8a37-4ce5-beb5-433d5c81961f\") " pod="opendatahub/maas-controller-5b7565664c-cprnk" Jul 13 07:15:18.120791 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:18.120711 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/720b787b-8a37-4ce5-beb5-433d5c81961f-webhook-cert\") pod \"maas-controller-5b7565664c-cprnk\" (UID: \"720b787b-8a37-4ce5-beb5-433d5c81961f\") " pod="opendatahub/maas-controller-5b7565664c-cprnk" Jul 13 07:15:18.221247 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:18.221222 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkbzr\" (UniqueName: \"kubernetes.io/projected/720b787b-8a37-4ce5-beb5-433d5c81961f-kube-api-access-pkbzr\") pod \"maas-controller-5b7565664c-cprnk\" (UID: \"720b787b-8a37-4ce5-beb5-433d5c81961f\") " pod="opendatahub/maas-controller-5b7565664c-cprnk" Jul 13 07:15:18.221347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:18.221317 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/720b787b-8a37-4ce5-beb5-433d5c81961f-webhook-cert\") pod \"maas-controller-5b7565664c-cprnk\" (UID: \"720b787b-8a37-4ce5-beb5-433d5c81961f\") " pod="opendatahub/maas-controller-5b7565664c-cprnk" Jul 13 07:15:18.223660 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:18.223639 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/720b787b-8a37-4ce5-beb5-433d5c81961f-webhook-cert\") pod \"maas-controller-5b7565664c-cprnk\" (UID: \"720b787b-8a37-4ce5-beb5-433d5c81961f\") " pod="opendatahub/maas-controller-5b7565664c-cprnk" Jul 13 07:15:18.228903 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:18.228882 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkbzr\" (UniqueName: \"kubernetes.io/projected/720b787b-8a37-4ce5-beb5-433d5c81961f-kube-api-access-pkbzr\") pod \"maas-controller-5b7565664c-cprnk\" (UID: \"720b787b-8a37-4ce5-beb5-433d5c81961f\") " pod="opendatahub/maas-controller-5b7565664c-cprnk" Jul 13 07:15:18.276651 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:18.276630 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7565664c-cprnk" Jul 13 07:15:18.408411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:18.408386 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7565664c-cprnk"] Jul 13 07:15:18.410500 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:15:18.410469 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720b787b_8a37_4ce5_beb5_433d5c81961f.slice/crio-61329e5dd024ad6a967bf0c645117216451eeb8968a60d49813efc9860470dd4 WatchSource:0}: Error finding container 61329e5dd024ad6a967bf0c645117216451eeb8968a60d49813efc9860470dd4: Status 404 returned error can't find the container with id 61329e5dd024ad6a967bf0c645117216451eeb8968a60d49813efc9860470dd4 Jul 13 07:15:18.411810 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:18.411794 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:15:19.203727 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:19.203645 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7565664c-cprnk" event={"ID":"720b787b-8a37-4ce5-beb5-433d5c81961f","Type":"ContainerStarted","Data":"9c37345779717ef428c11999fdc9f0c7b4b884ea21e033081d14743118cc8807"} Jul 13 07:15:19.203727 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:19.203698 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7565664c-cprnk" event={"ID":"720b787b-8a37-4ce5-beb5-433d5c81961f","Type":"ContainerStarted","Data":"61329e5dd024ad6a967bf0c645117216451eeb8968a60d49813efc9860470dd4"} Jul 13 07:15:19.204108 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:19.203819 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b7565664c-cprnk" Jul 13 07:15:19.222170 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:19.222126 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b7565664c-cprnk" podStartSLOduration=1.7606113410000002 podStartE2EDuration="2.22211177s" totalImagesPullingTime="461.500429ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:15:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:15:19.220585979 +0000 UTC m=+772.395089786" watchObservedRunningTime="2026-07-13 07:15:19.22211177 +0000 UTC m=+772.396615577" Jul 13 07:15:30.214600 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:30.214569 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b7565664c-cprnk" Jul 13 07:15:41.859606 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:41.859568 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:15:41.941175 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:15:41.941131 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:16:00.610143 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:16:00.610063 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:16:00.713551 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:16:00.713516 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:16:00.807179 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:16:00.807151 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:16:06.483780 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:16:06.483746 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:16:06.590774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:16:06.590731 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:16:06.686988 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:16:06.686960 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:16:06.835062 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:16:06.834986 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:16:12.659165 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:16:12.659124 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:16:12.760036 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:16:12.759997 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:16:17.450689 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:16:17.450656 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:18:52.665073 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:18:52.664985 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:21:50.665691 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:21:50.665593 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:21:50.724315 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:21:50.724285 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:24:02.024850 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:24:02.024819 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:24:03.497602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:24:03.497550 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:25:33.992192 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:25:33.992152 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:02.216848 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:02.216809 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:02.275306 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:02.275278 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:07.606358 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.606325 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:07.673317 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.673292 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:07.788010 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.787980 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv"] Jul 13 07:27:07.791727 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.791706 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.794245 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.794226 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d86b647c-openshift-default-dockercfg-mgl67\"" Jul 13 07:27:07.813589 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.813569 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv"] Jul 13 07:27:07.823573 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.823550 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:07.883897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.883824 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:07.897119 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.897094 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f793108-377c-44f9-ad19-2ffd2f9dddae-istiod-ca-cert\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.897254 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.897134 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-token\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.897254 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.897156 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-envoy\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.897254 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.897233 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-podinfo\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.897402 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.897289 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-socket\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.897402 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.897349 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-credential-socket\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.897402 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.897388 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdfmk\" (UniqueName: \"kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-kube-api-access-bdfmk\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.897521 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.897461 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-certs\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.897521 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.897484 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-data\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998168 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998141 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-certs\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998286 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998175 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-data\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998286 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998227 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f793108-377c-44f9-ad19-2ffd2f9dddae-istiod-ca-cert\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998286 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-token\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998286 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998284 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-envoy\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998477 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998303 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-podinfo\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998477 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998339 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-socket\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998477 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998376 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-credential-socket\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998477 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998415 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdfmk\" (UniqueName: \"kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-kube-api-access-bdfmk\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998671 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998617 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-data\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998671 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998635 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-certs\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998772 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998705 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-socket\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.998862 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.998841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-credential-socket\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:07.999119 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:07.999086 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f793108-377c-44f9-ad19-2ffd2f9dddae-istiod-ca-cert\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:08.000674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.000645 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-envoy\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:08.001800 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.001769 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-podinfo\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:08.008427 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.008401 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdfmk\" (UniqueName: \"kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-kube-api-access-bdfmk\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:08.008842 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.008822 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-token\") pod \"e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:08.102676 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.102656 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:08.240444 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.240415 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv"] Jul 13 07:27:08.242311 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:27:08.242286 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f793108_377c_44f9_ad19_2ffd2f9dddae.slice/crio-44ea170c441c5c9844e5c61b5e6e93d16bba3ad12f7675f97e2f9c7a4aa9ad6c WatchSource:0}: Error finding container 44ea170c441c5c9844e5c61b5e6e93d16bba3ad12f7675f97e2f9c7a4aa9ad6c: Status 404 returned error can't find the container with id 44ea170c441c5c9844e5c61b5e6e93d16bba3ad12f7675f97e2f9c7a4aa9ad6c Jul 13 07:27:08.244794 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.244764 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:27:08.244871 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.244827 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:27:08.244871 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.244854 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:27:08.796159 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.796128 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" event={"ID":"3f793108-377c-44f9-ad19-2ffd2f9dddae","Type":"ContainerStarted","Data":"13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f"} Jul 13 07:27:08.796567 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.796174 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" event={"ID":"3f793108-377c-44f9-ad19-2ffd2f9dddae","Type":"ContainerStarted","Data":"44ea170c441c5c9844e5c61b5e6e93d16bba3ad12f7675f97e2f9c7a4aa9ad6c"} Jul 13 07:27:08.817518 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.817472 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" podStartSLOduration=1.8174566589999999 podStartE2EDuration="1.817456659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:27:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:27:08.816491775 +0000 UTC m=+1481.990995582" watchObservedRunningTime="2026-07-13 07:27:08.817456659 +0000 UTC m=+1481.991960465" Jul 13 07:27:08.978750 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:08.978720 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:09.050934 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:09.050831 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:09.103575 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:09.103545 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:09.107770 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:09.107748 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:09.118004 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:09.117976 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:09.381689 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:09.381606 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:09.612633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:09.612606 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:09.714322 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:09.714300 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:09.800920 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:09.800893 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:09.801675 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:09.801653 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:11.386186 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:11.386155 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:14.002228 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:14.000613 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:14.182773 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:14.182739 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv"] Jul 13 07:27:14.183540 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:14.183193 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" podUID="3f793108-377c-44f9-ad19-2ffd2f9dddae" containerName="istio-proxy" containerID="cri-o://13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f" gracePeriod=30 Jul 13 07:27:14.196651 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:14.196613 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:14.300013 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:14.299910 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:14.471935 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:14.471905 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:16.046706 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:16.046665 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:20.027288 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.027238 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:20.040581 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.040536 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:20.114436 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114410 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-token\") pod \"3f793108-377c-44f9-ad19-2ffd2f9dddae\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " Jul 13 07:27:20.114548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114448 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-envoy\") pod \"3f793108-377c-44f9-ad19-2ffd2f9dddae\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " Jul 13 07:27:20.114548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114465 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-credential-socket\") pod \"3f793108-377c-44f9-ad19-2ffd2f9dddae\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " Jul 13 07:27:20.114548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114500 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-certs\") pod \"3f793108-377c-44f9-ad19-2ffd2f9dddae\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " Jul 13 07:27:20.114548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114527 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-podinfo\") pod \"3f793108-377c-44f9-ad19-2ffd2f9dddae\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " Jul 13 07:27:20.114757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114558 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdfmk\" (UniqueName: \"kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-kube-api-access-bdfmk\") pod \"3f793108-377c-44f9-ad19-2ffd2f9dddae\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " Jul 13 07:27:20.114757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114585 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-socket\") pod \"3f793108-377c-44f9-ad19-2ffd2f9dddae\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " Jul 13 07:27:20.114757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114696 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f793108-377c-44f9-ad19-2ffd2f9dddae-istiod-ca-cert\") pod \"3f793108-377c-44f9-ad19-2ffd2f9dddae\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " Jul 13 07:27:20.114757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114732 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-data\") pod \"3f793108-377c-44f9-ad19-2ffd2f9dddae\" (UID: \"3f793108-377c-44f9-ad19-2ffd2f9dddae\") " Jul 13 07:27:20.114969 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114732 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f793108-377c-44f9-ad19-2ffd2f9dddae" (UID: "3f793108-377c-44f9-ad19-2ffd2f9dddae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:20.114969 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114812 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f793108-377c-44f9-ad19-2ffd2f9dddae" (UID: "3f793108-377c-44f9-ad19-2ffd2f9dddae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:20.114969 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.114931 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f793108-377c-44f9-ad19-2ffd2f9dddae" (UID: "3f793108-377c-44f9-ad19-2ffd2f9dddae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:20.115136 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.115088 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:20.115136 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.115108 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:20.115136 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.115116 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:20.115136 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.115087 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f793108-377c-44f9-ad19-2ffd2f9dddae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f793108-377c-44f9-ad19-2ffd2f9dddae" (UID: "3f793108-377c-44f9-ad19-2ffd2f9dddae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:27:20.115602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.115582 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f793108-377c-44f9-ad19-2ffd2f9dddae" (UID: "3f793108-377c-44f9-ad19-2ffd2f9dddae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:20.116786 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.116763 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f793108-377c-44f9-ad19-2ffd2f9dddae" (UID: "3f793108-377c-44f9-ad19-2ffd2f9dddae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:27:20.116879 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.116823 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-kube-api-access-bdfmk" (OuterVolumeSpecName: "kube-api-access-bdfmk") pod "3f793108-377c-44f9-ad19-2ffd2f9dddae" (UID: "3f793108-377c-44f9-ad19-2ffd2f9dddae"). InnerVolumeSpecName "kube-api-access-bdfmk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:27:20.116951 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.116932 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f793108-377c-44f9-ad19-2ffd2f9dddae" (UID: "3f793108-377c-44f9-ad19-2ffd2f9dddae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:27:20.117017 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.117003 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f793108-377c-44f9-ad19-2ffd2f9dddae" (UID: "3f793108-377c-44f9-ad19-2ffd2f9dddae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:20.130090 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.130071 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:20.165897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.165839 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24"] Jul 13 07:27:20.166548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.166532 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f793108-377c-44f9-ad19-2ffd2f9dddae" containerName="istio-proxy" Jul 13 07:27:20.166595 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.166552 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f793108-377c-44f9-ad19-2ffd2f9dddae" containerName="istio-proxy" Jul 13 07:27:20.166658 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.166647 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f793108-377c-44f9-ad19-2ffd2f9dddae" containerName="istio-proxy" Jul 13 07:27:20.169308 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.169291 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.172325 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.172304 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d5942115-openshift-default-dockercfg-w66nf\"" Jul 13 07:27:20.180897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.180662 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24"] Jul 13 07:27:20.216124 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216092 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03b5e748-b753-42ba-9cbb-2a37e80795e9-istiod-ca-cert\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.216257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216133 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-socket\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.216257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216157 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-credential-socket\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.216257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216229 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-envoy\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.216381 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216264 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-data\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.216381 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216284 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb7jq\" (UniqueName: \"kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-kube-api-access-sb7jq\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.216381 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216350 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-certs\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.216475 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216407 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-token\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.216475 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216443 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-podinfo\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.216538 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216520 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:20.216538 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216531 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdfmk\" (UniqueName: \"kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-kube-api-access-bdfmk\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:20.216602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216541 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f793108-377c-44f9-ad19-2ffd2f9dddae-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:20.216602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216551 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:20.216602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216559 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:20.216602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.216570 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f793108-377c-44f9-ad19-2ffd2f9dddae-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:20.227877 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.227854 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:20.317781 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.317756 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-envoy\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.317929 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.317787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-data\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.317929 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.317805 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb7jq\" (UniqueName: \"kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-kube-api-access-sb7jq\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.317929 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.317850 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-certs\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.317929 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.317906 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-token\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.318151 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.317933 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-podinfo\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.318151 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.317987 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03b5e748-b753-42ba-9cbb-2a37e80795e9-istiod-ca-cert\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.318151 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.318013 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-socket\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.318151 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.318049 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-credential-socket\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.318454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.318193 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-data\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.318454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.318313 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-certs\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.318454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.318397 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-credential-socket\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.318454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.318400 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-socket\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.318845 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.318824 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03b5e748-b753-42ba-9cbb-2a37e80795e9-istiod-ca-cert\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.320153 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.320133 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-envoy\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.320399 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.320382 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-podinfo\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.325879 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.325859 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-token\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.326259 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.326236 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb7jq\" (UniqueName: \"kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-kube-api-access-sb7jq\") pod \"e2e-ait-d5942115-openshift-default-64d54d6865-9lb24\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.327244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.327199 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:20.481538 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.481508 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:20.610461 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.610357 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24"] Jul 13 07:27:20.612695 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:27:20.612671 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b5e748_b753_42ba_9cbb_2a37e80795e9.slice/crio-7f5518de172d8b6dd225ea6e8ccca48238c9bf927b8598fe5e94a0164e20cb26 WatchSource:0}: Error finding container 7f5518de172d8b6dd225ea6e8ccca48238c9bf927b8598fe5e94a0164e20cb26: Status 404 returned error can't find the container with id 7f5518de172d8b6dd225ea6e8ccca48238c9bf927b8598fe5e94a0164e20cb26 Jul 13 07:27:20.614772 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.614742 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:27:20.614858 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.614806 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:27:20.614858 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.614835 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:27:20.865075 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.864990 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" event={"ID":"03b5e748-b753-42ba-9cbb-2a37e80795e9","Type":"ContainerStarted","Data":"9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f"} Jul 13 07:27:20.865075 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.865052 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" event={"ID":"03b5e748-b753-42ba-9cbb-2a37e80795e9","Type":"ContainerStarted","Data":"7f5518de172d8b6dd225ea6e8ccca48238c9bf927b8598fe5e94a0164e20cb26"} Jul 13 07:27:20.866456 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.866432 2583 generic.go:356] "Generic (PLEG): container finished" podID="3f793108-377c-44f9-ad19-2ffd2f9dddae" containerID="13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f" exitCode=0 Jul 13 07:27:20.866580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.866457 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" event={"ID":"3f793108-377c-44f9-ad19-2ffd2f9dddae","Type":"ContainerDied","Data":"13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f"} Jul 13 07:27:20.866580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.866494 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" Jul 13 07:27:20.866580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.866514 2583 scope.go:117] "RemoveContainer" containerID="13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f" Jul 13 07:27:20.866715 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.866503 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv" event={"ID":"3f793108-377c-44f9-ad19-2ffd2f9dddae","Type":"ContainerDied","Data":"44ea170c441c5c9844e5c61b5e6e93d16bba3ad12f7675f97e2f9c7a4aa9ad6c"} Jul 13 07:27:20.876061 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.876045 2583 scope.go:117] "RemoveContainer" containerID="13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f" Jul 13 07:27:20.876394 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:27:20.876369 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f\": container with ID starting with 13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f not found: ID does not exist" containerID="13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f" Jul 13 07:27:20.876470 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.876401 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f"} err="failed to get container status \"13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f\": rpc error: code = NotFound desc = could not find container \"13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f\": container with ID starting with 13fd14384b0193526b3b1448f22be432f3f97581d7c7fb953e419a0ca02ea42f not found: ID does not exist" Jul 13 07:27:20.897868 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.897818 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" podStartSLOduration=0.897805718 podStartE2EDuration="897.805718ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:27:20.895909694 +0000 UTC m=+1494.070413540" watchObservedRunningTime="2026-07-13 07:27:20.897805718 +0000 UTC m=+1494.072309524" Jul 13 07:27:20.928925 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.928904 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv"] Jul 13 07:27:20.937255 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:20.937225 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d86b647c-openshift-default-7f7568fd4c-gqwrv"] Jul 13 07:27:21.285094 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:21.285063 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:21.385439 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:21.385415 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:21.429495 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:21.429462 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f793108-377c-44f9-ad19-2ffd2f9dddae" path="/var/lib/kubelet/pods/3f793108-377c-44f9-ad19-2ffd2f9dddae/volumes" Jul 13 07:27:21.482368 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:21.482343 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:21.486656 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:21.486635 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:21.487362 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:21.487341 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:21.604699 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:21.604631 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:21.872312 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:21.872240 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:21.873023 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:21.873006 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:22.030395 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:22.030364 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:22.131578 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:22.131515 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:23.494714 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:23.494683 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:26.398721 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:26.398685 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:26.812370 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:26.812340 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:32.471349 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:32.471317 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24"] Jul 13 07:27:32.471737 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:32.471487 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" podUID="03b5e748-b753-42ba-9cbb-2a37e80795e9" containerName="istio-proxy" containerID="cri-o://9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f" gracePeriod=30 Jul 13 07:27:32.495734 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:32.495701 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:32.659089 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:32.659059 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:34.601492 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:34.601463 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:36.875173 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:36.875139 2583 patch_prober.go:28] interesting pod/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 13 07:27:36.875574 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:36.875229 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" podUID="03b5e748-b753-42ba-9cbb-2a37e80795e9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 07:27:37.722296 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.722269 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:37.867039 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.866961 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-certs\") pod \"03b5e748-b753-42ba-9cbb-2a37e80795e9\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " Jul 13 07:27:37.867039 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.866996 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-credential-socket\") pod \"03b5e748-b753-42ba-9cbb-2a37e80795e9\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " Jul 13 07:27:37.867265 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867084 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-podinfo\") pod \"03b5e748-b753-42ba-9cbb-2a37e80795e9\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " Jul 13 07:27:37.867265 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867122 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sb7jq\" (UniqueName: \"kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-kube-api-access-sb7jq\") pod \"03b5e748-b753-42ba-9cbb-2a37e80795e9\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " Jul 13 07:27:37.867265 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867151 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-token\") pod \"03b5e748-b753-42ba-9cbb-2a37e80795e9\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " Jul 13 07:27:37.867265 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867177 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-envoy\") pod \"03b5e748-b753-42ba-9cbb-2a37e80795e9\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " Jul 13 07:27:37.867265 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867187 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "03b5e748-b753-42ba-9cbb-2a37e80795e9" (UID: "03b5e748-b753-42ba-9cbb-2a37e80795e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:37.867545 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867285 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-data\") pod \"03b5e748-b753-42ba-9cbb-2a37e80795e9\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " Jul 13 07:27:37.867545 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867323 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03b5e748-b753-42ba-9cbb-2a37e80795e9-istiod-ca-cert\") pod \"03b5e748-b753-42ba-9cbb-2a37e80795e9\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " Jul 13 07:27:37.867545 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867324 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "03b5e748-b753-42ba-9cbb-2a37e80795e9" (UID: "03b5e748-b753-42ba-9cbb-2a37e80795e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:37.867545 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867390 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-socket\") pod \"03b5e748-b753-42ba-9cbb-2a37e80795e9\" (UID: \"03b5e748-b753-42ba-9cbb-2a37e80795e9\") " Jul 13 07:27:37.867767 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867714 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-data" (OuterVolumeSpecName: "istio-data") pod "03b5e748-b753-42ba-9cbb-2a37e80795e9" (UID: "03b5e748-b753-42ba-9cbb-2a37e80795e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:37.867830 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867806 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "03b5e748-b753-42ba-9cbb-2a37e80795e9" (UID: "03b5e748-b753-42ba-9cbb-2a37e80795e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:37.867910 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867891 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:37.867963 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867905 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03b5e748-b753-42ba-9cbb-2a37e80795e9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "03b5e748-b753-42ba-9cbb-2a37e80795e9" (UID: "03b5e748-b753-42ba-9cbb-2a37e80795e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:27:37.867963 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867915 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:37.867963 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.867930 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:37.869447 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.869422 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-token" (OuterVolumeSpecName: "istio-token") pod "03b5e748-b753-42ba-9cbb-2a37e80795e9" (UID: "03b5e748-b753-42ba-9cbb-2a37e80795e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:27:37.869562 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.869537 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "03b5e748-b753-42ba-9cbb-2a37e80795e9" (UID: "03b5e748-b753-42ba-9cbb-2a37e80795e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:37.869717 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.869690 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-kube-api-access-sb7jq" (OuterVolumeSpecName: "kube-api-access-sb7jq") pod "03b5e748-b753-42ba-9cbb-2a37e80795e9" (UID: "03b5e748-b753-42ba-9cbb-2a37e80795e9"). InnerVolumeSpecName "kube-api-access-sb7jq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:27:37.869804 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.869789 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "03b5e748-b753-42ba-9cbb-2a37e80795e9" (UID: "03b5e748-b753-42ba-9cbb-2a37e80795e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:27:37.952295 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.952253 2583 generic.go:356] "Generic (PLEG): container finished" podID="03b5e748-b753-42ba-9cbb-2a37e80795e9" containerID="9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f" exitCode=0 Jul 13 07:27:37.952633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.952305 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" Jul 13 07:27:37.952633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.952338 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" event={"ID":"03b5e748-b753-42ba-9cbb-2a37e80795e9","Type":"ContainerDied","Data":"9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f"} Jul 13 07:27:37.952633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.952388 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24" event={"ID":"03b5e748-b753-42ba-9cbb-2a37e80795e9","Type":"ContainerDied","Data":"7f5518de172d8b6dd225ea6e8ccca48238c9bf927b8598fe5e94a0164e20cb26"} Jul 13 07:27:37.952633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.952409 2583 scope.go:117] "RemoveContainer" containerID="9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f" Jul 13 07:27:37.964984 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.964966 2583 scope.go:117] "RemoveContainer" containerID="9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f" Jul 13 07:27:37.965324 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:27:37.965306 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f\": container with ID starting with 9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f not found: ID does not exist" containerID="9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f" Jul 13 07:27:37.965398 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.965336 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f"} err="failed to get container status \"9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f\": rpc error: code = NotFound desc = could not find container \"9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f\": container with ID starting with 9a0a5b2bc3d1ce5a6a1c4787482c2df811a42fac0b22c636db35414dbfd58a1f not found: ID does not exist" Jul 13 07:27:37.968779 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.968761 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:37.968852 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.968782 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb7jq\" (UniqueName: \"kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-kube-api-access-sb7jq\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:37.968852 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.968793 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:37.968852 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.968801 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:37.968852 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.968808 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03b5e748-b753-42ba-9cbb-2a37e80795e9-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:37.968852 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.968816 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5e748-b753-42ba-9cbb-2a37e80795e9-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:37.975073 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.975052 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24"] Jul 13 07:27:37.979271 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:37.979252 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d5942115-openshift-default-64d54d6865-9lb24"] Jul 13 07:27:38.608272 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.608229 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:38.695224 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.695183 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:38.784056 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.784027 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs"] Jul 13 07:27:38.784482 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.784457 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03b5e748-b753-42ba-9cbb-2a37e80795e9" containerName="istio-proxy" Jul 13 07:27:38.784482 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.784481 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="03b5e748-b753-42ba-9cbb-2a37e80795e9" containerName="istio-proxy" Jul 13 07:27:38.784668 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.784556 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="03b5e748-b753-42ba-9cbb-2a37e80795e9" containerName="istio-proxy" Jul 13 07:27:38.787432 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.787410 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.789953 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.789929 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-629e4f78-openshift-default-dockercfg-sfj4h\"" Jul 13 07:27:38.800230 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.800175 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs"] Jul 13 07:27:38.824923 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.824895 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:38.876251 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.876176 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1af66b-620d-481a-93ca-024e93268d03-istio-podinfo\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.876339 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.876267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-credential-socket\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.876339 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.876305 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-istio-token\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.876339 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.876331 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1af66b-620d-481a-93ca-024e93268d03-istiod-ca-cert\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.876450 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.876382 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-data\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.876450 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.876411 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66ggm\" (UniqueName: \"kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-kube-api-access-66ggm\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.876450 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.876428 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-envoy\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.876568 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.876461 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-socket\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.876568 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.876489 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-certs\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.924363 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.924338 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:38.977682 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.977656 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-data\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.977693 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66ggm\" (UniqueName: \"kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-kube-api-access-66ggm\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.977717 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-envoy\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.977748 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-socket\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.977770 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-certs\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.977876 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1af66b-620d-481a-93ca-024e93268d03-istio-podinfo\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.977922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-credential-socket\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.977960 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-istio-token\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.978012 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1af66b-620d-481a-93ca-024e93268d03-istiod-ca-cert\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978518 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.978099 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-data\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978518 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.978151 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-socket\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978518 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.978228 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-certs\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978518 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.978444 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-credential-socket\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.978968 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.978946 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1af66b-620d-481a-93ca-024e93268d03-istiod-ca-cert\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.979998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.979970 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-envoy\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.980228 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.980187 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1af66b-620d-481a-93ca-024e93268d03-istio-podinfo\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.987847 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.987823 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66ggm\" (UniqueName: \"kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-kube-api-access-66ggm\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:38.988157 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:38.988135 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-istio-token\") pod \"e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:39.098376 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.098352 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:39.233773 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.233748 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs"] Jul 13 07:27:39.235637 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:27:39.235609 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1af66b_620d_481a_93ca_024e93268d03.slice/crio-e31348e8e1158c216d41d44a5068ac2d4404d752b2cc2f786ed9c3e8920f4e83 WatchSource:0}: Error finding container e31348e8e1158c216d41d44a5068ac2d4404d752b2cc2f786ed9c3e8920f4e83: Status 404 returned error can't find the container with id e31348e8e1158c216d41d44a5068ac2d4404d752b2cc2f786ed9c3e8920f4e83 Jul 13 07:27:39.238293 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.238262 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:27:39.238371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.238324 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:27:39.238371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.238353 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:27:39.428977 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.428895 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03b5e748-b753-42ba-9cbb-2a37e80795e9" path="/var/lib/kubelet/pods/03b5e748-b753-42ba-9cbb-2a37e80795e9/volumes" Jul 13 07:27:39.912298 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.912262 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:39.966182 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.966143 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" event={"ID":"fd1af66b-620d-481a-93ca-024e93268d03","Type":"ContainerStarted","Data":"43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312"} Jul 13 07:27:39.966376 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.966229 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" event={"ID":"fd1af66b-620d-481a-93ca-024e93268d03","Type":"ContainerStarted","Data":"e31348e8e1158c216d41d44a5068ac2d4404d752b2cc2f786ed9c3e8920f4e83"} Jul 13 07:27:39.975681 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.975646 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9"] Jul 13 07:27:39.978689 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.978665 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:39.981466 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.981447 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-629e4f78-serving-cert\"" Jul 13 07:27:39.990932 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.990912 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9"] Jul 13 07:27:39.994700 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.994658 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" podStartSLOduration=1.994644286 podStartE2EDuration="1.994644286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:27:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:27:39.993551357 +0000 UTC m=+1513.168055165" watchObservedRunningTime="2026-07-13 07:27:39.994644286 +0000 UTC m=+1513.169148092" Jul 13 07:27:39.997191 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:39.996973 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:40.058999 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.058974 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:40.089476 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.089445 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-maas-api-tls\") pod \"maas-api-e2e-derive-629e4f78-59785585b-6rtm9\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.089638 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.089520 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd9ll\" (UniqueName: \"kubernetes.io/projected/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-kube-api-access-pd9ll\") pod \"maas-api-e2e-derive-629e4f78-59785585b-6rtm9\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.089720 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.089700 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-openshift-service-ca\") pod \"maas-api-e2e-derive-629e4f78-59785585b-6rtm9\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.099246 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.099221 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:40.104040 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.104012 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:40.190819 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.190731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-maas-api-tls\") pod \"maas-api-e2e-derive-629e4f78-59785585b-6rtm9\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.190819 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.190787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd9ll\" (UniqueName: \"kubernetes.io/projected/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-kube-api-access-pd9ll\") pod \"maas-api-e2e-derive-629e4f78-59785585b-6rtm9\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.191026 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.190829 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-openshift-service-ca\") pod \"maas-api-e2e-derive-629e4f78-59785585b-6rtm9\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.191495 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.191470 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-openshift-service-ca\") pod \"maas-api-e2e-derive-629e4f78-59785585b-6rtm9\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.193147 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.193125 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-maas-api-tls\") pod \"maas-api-e2e-derive-629e4f78-59785585b-6rtm9\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.198862 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.198838 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd9ll\" (UniqueName: \"kubernetes.io/projected/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-kube-api-access-pd9ll\") pod \"maas-api-e2e-derive-629e4f78-59785585b-6rtm9\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.301306 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.301277 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.436108 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.436017 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9"] Jul 13 07:27:40.438881 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:27:40.438852 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2dd501_3ec2_4831_b0d6_49f2665be9f0.slice/crio-3c610b0c46a0caf2f628f83205588b7dcf249c506f2830f8e2ebec5d209dcec0 WatchSource:0}: Error finding container 3c610b0c46a0caf2f628f83205588b7dcf249c506f2830f8e2ebec5d209dcec0: Status 404 returned error can't find the container with id 3c610b0c46a0caf2f628f83205588b7dcf249c506f2830f8e2ebec5d209dcec0 Jul 13 07:27:40.440337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.440317 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:27:40.446803 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.446780 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:40.553811 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.553787 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:40.972525 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.972420 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" event={"ID":"7e2dd501-3ec2-4831-b0d6-49f2665be9f0","Type":"ContainerStarted","Data":"f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a"} Jul 13 07:27:40.972525 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.972491 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" event={"ID":"7e2dd501-3ec2-4831-b0d6-49f2665be9f0","Type":"ContainerStarted","Data":"3c610b0c46a0caf2f628f83205588b7dcf249c506f2830f8e2ebec5d209dcec0"} Jul 13 07:27:40.973228 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.973187 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:40.973348 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.973317 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:40.974047 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.974031 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:40.991350 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:40.991311 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" podStartSLOduration=1.717627351 podStartE2EDuration="1.991292886s" totalImagesPullingTime="273.665535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:27:40.988423789 +0000 UTC m=+1514.162927609" watchObservedRunningTime="2026-07-13 07:27:40.991292886 +0000 UTC m=+1514.165796694" Jul 13 07:27:41.885719 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:41.885660 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:44.563865 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:44.563832 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:44.679464 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:44.679413 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs"] Jul 13 07:27:44.679659 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:44.679606 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" podUID="fd1af66b-620d-481a-93ca-024e93268d03" containerName="istio-proxy" containerID="cri-o://43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312" gracePeriod=30 Jul 13 07:27:44.697080 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:44.697052 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:44.802161 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:44.802108 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:44.901281 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:44.898094 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9"] Jul 13 07:27:44.901281 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:44.898668 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" podUID="7e2dd501-3ec2-4831-b0d6-49f2665be9f0" containerName="maas-api" containerID="cri-o://f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a" gracePeriod=30 Jul 13 07:27:44.925386 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:44.925357 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:45.164540 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:45.164479 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:45.254453 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:45.254420 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-maas-api-tls\") pod \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " Jul 13 07:27:45.254601 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:45.254522 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd9ll\" (UniqueName: \"kubernetes.io/projected/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-kube-api-access-pd9ll\") pod \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " Jul 13 07:27:45.254601 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:45.254550 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-openshift-service-ca\") pod \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\" (UID: \"7e2dd501-3ec2-4831-b0d6-49f2665be9f0\") " Jul 13 07:27:45.254964 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:45.254938 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7e2dd501-3ec2-4831-b0d6-49f2665be9f0" (UID: "7e2dd501-3ec2-4831-b0d6-49f2665be9f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:27:45.256564 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:45.256543 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7e2dd501-3ec2-4831-b0d6-49f2665be9f0" (UID: "7e2dd501-3ec2-4831-b0d6-49f2665be9f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:27:45.256672 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:45.256647 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-kube-api-access-pd9ll" (OuterVolumeSpecName: "kube-api-access-pd9ll") pod "7e2dd501-3ec2-4831-b0d6-49f2665be9f0" (UID: "7e2dd501-3ec2-4831-b0d6-49f2665be9f0"). InnerVolumeSpecName "kube-api-access-pd9ll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:27:45.355548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:45.355519 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-maas-api-tls\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:45.355548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:45.355541 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd9ll\" (UniqueName: \"kubernetes.io/projected/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-kube-api-access-pd9ll\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:45.355548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:45.355552 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e2dd501-3ec2-4831-b0d6-49f2665be9f0-openshift-service-ca\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:46.011737 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:46.011707 2583 generic.go:356] "Generic (PLEG): container finished" podID="7e2dd501-3ec2-4831-b0d6-49f2665be9f0" containerID="f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a" exitCode=0 Jul 13 07:27:46.012121 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:46.011745 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" Jul 13 07:27:46.012121 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:46.011794 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" event={"ID":"7e2dd501-3ec2-4831-b0d6-49f2665be9f0","Type":"ContainerDied","Data":"f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a"} Jul 13 07:27:46.012121 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:46.011843 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9" event={"ID":"7e2dd501-3ec2-4831-b0d6-49f2665be9f0","Type":"ContainerDied","Data":"3c610b0c46a0caf2f628f83205588b7dcf249c506f2830f8e2ebec5d209dcec0"} Jul 13 07:27:46.012121 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:46.011865 2583 scope.go:117] "RemoveContainer" containerID="f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a" Jul 13 07:27:46.021084 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:46.021069 2583 scope.go:117] "RemoveContainer" containerID="f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a" Jul 13 07:27:46.021407 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:27:46.021380 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a\": container with ID starting with f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a not found: ID does not exist" containerID="f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a" Jul 13 07:27:46.021465 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:46.021411 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a"} err="failed to get container status \"f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a\": rpc error: code = NotFound desc = could not find container \"f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a\": container with ID starting with f3518321afbb17035a0f99bfe9250ceb0bb42a7a4694b120bef029bec454c25a not found: ID does not exist" Jul 13 07:27:46.027985 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:46.027964 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9"] Jul 13 07:27:46.031901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:46.031882 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-629e4f78-59785585b-6rtm9"] Jul 13 07:27:46.564879 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:46.564853 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:47.428461 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:47.428431 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e2dd501-3ec2-4831-b0d6-49f2665be9f0" path="/var/lib/kubelet/pods/7e2dd501-3ec2-4831-b0d6-49f2665be9f0/volumes" Jul 13 07:27:50.537264 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.537233 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:50.636313 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.636287 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:50.856383 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.856359 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:50.908872 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.908848 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1af66b-620d-481a-93ca-024e93268d03-istiod-ca-cert\") pod \"fd1af66b-620d-481a-93ca-024e93268d03\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " Jul 13 07:27:50.908993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.908915 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-data\") pod \"fd1af66b-620d-481a-93ca-024e93268d03\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " Jul 13 07:27:50.908993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.908933 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-envoy\") pod \"fd1af66b-620d-481a-93ca-024e93268d03\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " Jul 13 07:27:50.908993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.908961 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-credential-socket\") pod \"fd1af66b-620d-481a-93ca-024e93268d03\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " Jul 13 07:27:50.908993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.908983 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-istio-token\") pod \"fd1af66b-620d-481a-93ca-024e93268d03\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " Jul 13 07:27:50.909193 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.909014 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-socket\") pod \"fd1af66b-620d-481a-93ca-024e93268d03\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " Jul 13 07:27:50.909193 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.909043 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1af66b-620d-481a-93ca-024e93268d03-istio-podinfo\") pod \"fd1af66b-620d-481a-93ca-024e93268d03\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " Jul 13 07:27:50.909193 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.909101 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66ggm\" (UniqueName: \"kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-kube-api-access-66ggm\") pod \"fd1af66b-620d-481a-93ca-024e93268d03\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " Jul 13 07:27:50.909193 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.909134 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-certs\") pod \"fd1af66b-620d-481a-93ca-024e93268d03\" (UID: \"fd1af66b-620d-481a-93ca-024e93268d03\") " Jul 13 07:27:50.909402 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.909232 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd1af66b-620d-481a-93ca-024e93268d03-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fd1af66b-620d-481a-93ca-024e93268d03" (UID: "fd1af66b-620d-481a-93ca-024e93268d03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:27:50.909536 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.909483 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fd1af66b-620d-481a-93ca-024e93268d03" (UID: "fd1af66b-620d-481a-93ca-024e93268d03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:50.909615 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.909517 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fd1af66b-620d-481a-93ca-024e93268d03" (UID: "fd1af66b-620d-481a-93ca-024e93268d03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:50.909615 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.909522 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1af66b-620d-481a-93ca-024e93268d03-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:50.909768 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.909746 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fd1af66b-620d-481a-93ca-024e93268d03" (UID: "fd1af66b-620d-481a-93ca-024e93268d03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:50.910224 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.910174 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-data" (OuterVolumeSpecName: "istio-data") pod "fd1af66b-620d-481a-93ca-024e93268d03" (UID: "fd1af66b-620d-481a-93ca-024e93268d03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:50.911342 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.911319 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fd1af66b-620d-481a-93ca-024e93268d03" (UID: "fd1af66b-620d-481a-93ca-024e93268d03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:27:50.911517 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.911494 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd1af66b-620d-481a-93ca-024e93268d03-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fd1af66b-620d-481a-93ca-024e93268d03" (UID: "fd1af66b-620d-481a-93ca-024e93268d03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:27:50.911701 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.911673 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd1af66b-620d-481a-93ca-024e93268d03" (UID: "fd1af66b-620d-481a-93ca-024e93268d03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:27:50.911793 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:50.911719 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-kube-api-access-66ggm" (OuterVolumeSpecName: "kube-api-access-66ggm") pod "fd1af66b-620d-481a-93ca-024e93268d03" (UID: "fd1af66b-620d-481a-93ca-024e93268d03"). InnerVolumeSpecName "kube-api-access-66ggm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:27:51.010859 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.010834 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66ggm\" (UniqueName: \"kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-kube-api-access-66ggm\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:51.010859 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.010858 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:51.011092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.010868 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:51.011092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.010876 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:51.011092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.010884 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:51.011092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.010892 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1af66b-620d-481a-93ca-024e93268d03-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:51.011092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.010900 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1af66b-620d-481a-93ca-024e93268d03-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:51.011092 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.010911 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1af66b-620d-481a-93ca-024e93268d03-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:27:51.014571 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.014527 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:51.051085 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.051059 2583 generic.go:356] "Generic (PLEG): container finished" podID="fd1af66b-620d-481a-93ca-024e93268d03" containerID="43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312" exitCode=0 Jul 13 07:27:51.051163 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.051106 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" Jul 13 07:27:51.051163 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.051141 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" event={"ID":"fd1af66b-620d-481a-93ca-024e93268d03","Type":"ContainerDied","Data":"43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312"} Jul 13 07:27:51.051258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.051192 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs" event={"ID":"fd1af66b-620d-481a-93ca-024e93268d03","Type":"ContainerDied","Data":"e31348e8e1158c216d41d44a5068ac2d4404d752b2cc2f786ed9c3e8920f4e83"} Jul 13 07:27:51.051258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.051228 2583 scope.go:117] "RemoveContainer" containerID="43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312" Jul 13 07:27:51.060639 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.060621 2583 scope.go:117] "RemoveContainer" containerID="43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312" Jul 13 07:27:51.060896 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:27:51.060876 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312\": container with ID starting with 43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312 not found: ID does not exist" containerID="43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312" Jul 13 07:27:51.060950 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.060906 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312"} err="failed to get container status \"43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312\": rpc error: code = NotFound desc = could not find container \"43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312\": container with ID starting with 43603a7c03a6a51a5f068755d68bb81af9d9e5ed0e8a678e764bea9919488312 not found: ID does not exist" Jul 13 07:27:51.077076 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.077051 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs"] Jul 13 07:27:51.085454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.085406 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-629e4f78-openshift-default-7d96fdb879-hzbhs"] Jul 13 07:27:51.089085 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.089066 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:51.142976 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.142951 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:51.429009 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:51.428936 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd1af66b-620d-481a-93ca-024e93268d03" path="/var/lib/kubelet/pods/fd1af66b-620d-481a-93ca-024e93268d03/volumes" Jul 13 07:27:52.560862 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:52.560814 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:59.002447 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:59.002410 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:59.112593 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:59.112547 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:59.263273 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:59.263161 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:59.384469 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:59.384443 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:27:59.743426 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:27:59.743389 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:01.908491 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:01.908455 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:25.702127 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:25.702095 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:31.678489 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:31.678453 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:31.777310 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:31.777279 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:31.877659 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:31.877634 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:41.004075 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:41.004033 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:41.070398 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:41.070374 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:47.839736 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:47.839702 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:53.864379 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:53.864344 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:53.949224 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:53.949167 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:28:54.049222 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:28:54.049181 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:04.608388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:04.608349 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:05.587837 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:05.587798 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:11.396717 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:11.396685 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:11.501586 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:11.501547 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:11.597818 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:11.597779 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:11.697713 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:11.697634 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:12.473004 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:12.472978 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:12.572802 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:12.572761 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:12.675392 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:12.675354 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:14.320212 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:14.320181 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:20.006232 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:20.006165 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:20.106480 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:20.106455 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:20.675813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:20.675779 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:26.514593 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:26.514560 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:26.615316 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:26.615288 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:26.824015 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:26.823938 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:27.074609 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.074536 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs"] Jul 13 07:29:27.074979 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.074966 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e2dd501-3ec2-4831-b0d6-49f2665be9f0" containerName="maas-api" Jul 13 07:29:27.075033 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.074981 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e2dd501-3ec2-4831-b0d6-49f2665be9f0" containerName="maas-api" Jul 13 07:29:27.075033 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.075007 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd1af66b-620d-481a-93ca-024e93268d03" containerName="istio-proxy" Jul 13 07:29:27.075033 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.075028 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd1af66b-620d-481a-93ca-024e93268d03" containerName="istio-proxy" Jul 13 07:29:27.075127 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.075093 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd1af66b-620d-481a-93ca-024e93268d03" containerName="istio-proxy" Jul 13 07:29:27.075127 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.075103 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e2dd501-3ec2-4831-b0d6-49f2665be9f0" containerName="maas-api" Jul 13 07:29:27.078303 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.078281 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.080879 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.080858 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2d7b36bf-openshift-default-dockercfg-tt2dp\"" Jul 13 07:29:27.099611 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.099585 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs"] Jul 13 07:29:27.100738 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.100715 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:27.196331 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.196297 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95xd2\" (UniqueName: \"kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-kube-api-access-95xd2\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.196483 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.196342 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-envoy\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.196483 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.196371 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-certs\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.196483 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.196441 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-podinfo\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.196615 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.196514 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-data\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.196615 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.196543 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-socket\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.196615 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.196574 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-token\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.196724 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.196617 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istiod-ca-cert\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.196724 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.196688 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-credential-socket\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298052 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-credential-socket\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298197 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298068 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95xd2\" (UniqueName: \"kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-kube-api-access-95xd2\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298197 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-envoy\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298197 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298125 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-certs\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298197 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298154 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-podinfo\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298469 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298232 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-data\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298469 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-socket\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298469 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298307 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-token\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298469 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298363 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istiod-ca-cert\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298542 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-data\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298605 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-certs\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298636 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-socket\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.298764 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.298737 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-credential-socket\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.299076 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.299056 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istiod-ca-cert\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.300313 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.300289 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-envoy\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.300507 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.300493 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-podinfo\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.306257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.306237 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-token\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.306922 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.306896 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95xd2\" (UniqueName: \"kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-kube-api-access-95xd2\") pod \"e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.391346 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.391288 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2d7b36bf-openshift-default-dockercfg-tt2dp\"" Jul 13 07:29:27.399418 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.399398 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:27.536447 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.536423 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs"] Jul 13 07:29:27.538185 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:29:27.538159 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f6ec77_ecf9_40f2_a23a_90efd3a9e4e5.slice/crio-c64df9b0f221c0406639171f08b1cde85523f88ce51157457cf847e71fa3c96a WatchSource:0}: Error finding container c64df9b0f221c0406639171f08b1cde85523f88ce51157457cf847e71fa3c96a: Status 404 returned error can't find the container with id c64df9b0f221c0406639171f08b1cde85523f88ce51157457cf847e71fa3c96a Jul 13 07:29:27.540306 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.540279 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:29:27.540392 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.540333 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:29:27.540392 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.540362 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:29:27.929726 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:27.929691 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:28.540040 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:28.540005 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" event={"ID":"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5","Type":"ContainerStarted","Data":"902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538"} Jul 13 07:29:28.540419 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:28.540090 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" event={"ID":"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5","Type":"ContainerStarted","Data":"c64df9b0f221c0406639171f08b1cde85523f88ce51157457cf847e71fa3c96a"} Jul 13 07:29:28.561983 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:28.561908 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" podStartSLOduration=1.5618936350000001 podStartE2EDuration="1.561893635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:29:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:29:28.560730845 +0000 UTC m=+1621.735234656" watchObservedRunningTime="2026-07-13 07:29:28.561893635 +0000 UTC m=+1621.736397441" Jul 13 07:29:29.400101 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:29.400073 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:29.404693 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:29.404671 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:29.544261 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:29.544241 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:29.544930 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:29.544913 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:30.476329 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.476287 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:30.540890 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.540859 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5"] Jul 13 07:29:30.544753 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.544729 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:30.547670 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.547645 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2d7b36bf-serving-cert\"" Jul 13 07:29:30.554706 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.554682 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5"] Jul 13 07:29:30.574853 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.574821 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:30.624543 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.624520 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-openshift-service-ca\") pod \"maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:30.624629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.624547 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-maas-api-tls\") pod \"maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:30.624691 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.624668 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmz86\" (UniqueName: \"kubernetes.io/projected/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-kube-api-access-jmz86\") pod \"maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:30.672251 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.672199 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:30.725150 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.725125 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-maas-api-tls\") pod \"maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:30.725277 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.725218 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmz86\" (UniqueName: \"kubernetes.io/projected/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-kube-api-access-jmz86\") pod \"maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:30.725325 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:29:30.725288 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-serving-cert: secret "maas-api-e2e-mt-2d7b36bf-serving-cert" not found Jul 13 07:29:30.725368 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.725330 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-openshift-service-ca\") pod \"maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:30.725368 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:29:30.725351 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-maas-api-tls podName:4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78 nodeName:}" failed. No retries permitted until 2026-07-13 07:29:31.225337018 +0000 UTC m=+1624.399840803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-maas-api-tls") pod "maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" (UID: "4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78") : secret "maas-api-e2e-mt-2d7b36bf-serving-cert" not found Jul 13 07:29:30.725916 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.725899 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-openshift-service-ca\") pod \"maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:30.734358 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.734304 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmz86\" (UniqueName: \"kubernetes.io/projected/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-kube-api-access-jmz86\") pod \"maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:30.898009 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:30.897977 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:31.050803 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:31.050727 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:31.149388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:31.149360 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:31.230104 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:31.230071 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-maas-api-tls\") pod \"maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:31.232454 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:31.232432 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-maas-api-tls\") pod \"maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:31.456825 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:31.456758 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:31.592993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:31.592948 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5"] Jul 13 07:29:31.593650 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:29:31.593620 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c67fd7d_56cb_4fa5_b8a2_0bfa2d5fcf78.slice/crio-acd51ae0a10b953b121a98e2ccd274ba135e26cb59e739b63815be14936d16b5 WatchSource:0}: Error finding container acd51ae0a10b953b121a98e2ccd274ba135e26cb59e739b63815be14936d16b5: Status 404 returned error can't find the container with id acd51ae0a10b953b121a98e2ccd274ba135e26cb59e739b63815be14936d16b5 Jul 13 07:29:32.562656 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:32.562621 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" event={"ID":"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78","Type":"ContainerStarted","Data":"05c9283e5804d1a2322ad4ee9b8adefc4fb81adb5483f9ab4165ce06ee3aace1"} Jul 13 07:29:32.562815 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:32.562682 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" event={"ID":"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78","Type":"ContainerStarted","Data":"acd51ae0a10b953b121a98e2ccd274ba135e26cb59e739b63815be14936d16b5"} Jul 13 07:29:32.562815 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:32.562720 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:32.583421 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:32.583380 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" podStartSLOduration=2.09552329 podStartE2EDuration="2.58336528s" totalImagesPullingTime="487.84199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:29:32.580053692 +0000 UTC m=+1625.754557523" watchObservedRunningTime="2026-07-13 07:29:32.58336528 +0000 UTC m=+1625.757869158" Jul 13 07:29:34.968610 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:34.968574 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:35.606833 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:35.606803 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:35.703818 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:35.703791 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:36.038867 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:36.038817 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:38.572822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:38.572797 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:41.650027 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:41.649988 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:41.713829 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:41.713790 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:42.158189 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.158146 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:42.498081 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.498049 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5"] Jul 13 07:29:42.498278 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.498254 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" podUID="4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78" containerName="maas-api" containerID="cri-o://05c9283e5804d1a2322ad4ee9b8adefc4fb81adb5483f9ab4165ce06ee3aace1" gracePeriod=30 Jul 13 07:29:42.520120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.520088 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:42.622266 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.622237 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:42.628523 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.628499 2583 generic.go:356] "Generic (PLEG): container finished" podID="4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78" containerID="05c9283e5804d1a2322ad4ee9b8adefc4fb81adb5483f9ab4165ce06ee3aace1" exitCode=0 Jul 13 07:29:42.628636 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.628561 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" event={"ID":"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78","Type":"ContainerDied","Data":"05c9283e5804d1a2322ad4ee9b8adefc4fb81adb5483f9ab4165ce06ee3aace1"} Jul 13 07:29:42.750952 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.750889 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:42.844149 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.844119 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmz86\" (UniqueName: \"kubernetes.io/projected/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-kube-api-access-jmz86\") pod \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " Jul 13 07:29:42.844317 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.844192 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-openshift-service-ca\") pod \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " Jul 13 07:29:42.844317 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.844259 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-maas-api-tls\") pod \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\" (UID: \"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78\") " Jul 13 07:29:42.844532 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.844503 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78" (UID: "4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:29:42.844689 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.844673 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-openshift-service-ca\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:42.846171 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.846150 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-kube-api-access-jmz86" (OuterVolumeSpecName: "kube-api-access-jmz86") pod "4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78" (UID: "4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78"). InnerVolumeSpecName "kube-api-access-jmz86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:29:42.846291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.846273 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78" (UID: "4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:29:42.945451 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.945422 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmz86\" (UniqueName: \"kubernetes.io/projected/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-kube-api-access-jmz86\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:42.945451 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:42.945448 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78-maas-api-tls\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:43.635398 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.635333 2583 generic.go:356] "Generic (PLEG): container finished" podID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerID="902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538" exitCode=0 Jul 13 07:29:43.635557 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.635407 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" event={"ID":"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5","Type":"ContainerDied","Data":"902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538"} Jul 13 07:29:43.635853 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.635837 2583 scope.go:117] "RemoveContainer" containerID="902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538" Jul 13 07:29:43.636785 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.636754 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" event={"ID":"4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78","Type":"ContainerDied","Data":"acd51ae0a10b953b121a98e2ccd274ba135e26cb59e739b63815be14936d16b5"} Jul 13 07:29:43.636785 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.636781 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5" Jul 13 07:29:43.636945 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.636812 2583 scope.go:117] "RemoveContainer" containerID="05c9283e5804d1a2322ad4ee9b8adefc4fb81adb5483f9ab4165ce06ee3aace1" Jul 13 07:29:43.636945 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.636891 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:29:43.637053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.636972 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:29:43.637053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.637020 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:29:43.677459 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.677436 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5"] Jul 13 07:29:43.686877 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:43.686855 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d7b36bf-5bd9694f9c-p4wf5"] Jul 13 07:29:44.544609 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:44.544575 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:44.644852 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:44.644813 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" event={"ID":"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5","Type":"ContainerStarted","Data":"3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897"} Jul 13 07:29:45.400359 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:45.400327 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:45.401683 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:45.401588 2583 patch_prober.go:28] interesting pod/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 13 07:29:45.401920 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:45.401894 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 13 07:29:45.429049 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:45.429026 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78" path="/var/lib/kubelet/pods/4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78/volumes" Jul 13 07:29:46.400872 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:46.400836 2583 patch_prober.go:28] interesting pod/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 13 07:29:46.401441 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:46.401410 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 13 07:29:47.402564 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:47.402529 2583 patch_prober.go:28] interesting pod/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 13 07:29:47.403106 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:47.403076 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 13 07:29:48.404463 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:48.404432 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:48.404924 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:48.404815 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:48.405384 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:48.405366 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:48.439225 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:48.439175 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:48.517793 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:48.517761 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:48.692482 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:48.692415 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs"] Jul 13 07:29:48.710144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:48.710121 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:49.032716 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:49.032682 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:49.132081 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:49.132059 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:49.240769 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:49.240748 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:49.672446 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:49.672417 2583 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" secret="" err="secret \"e2e-mt-2d7b36bf-openshift-default-dockercfg-tt2dp\" not found" Jul 13 07:29:50.677542 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:50.677509 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerName="istio-proxy" containerID="cri-o://3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897" gracePeriod=30 Jul 13 07:29:54.953487 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:54.953455 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:55.058911 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:55.058877 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:55.157796 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:55.157762 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:55.308806 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:55.308784 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:29:55.926649 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:55.926629 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:56.065249 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065222 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-token\") pod \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " Jul 13 07:29:56.065646 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065268 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-certs\") pod \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " Jul 13 07:29:56.065646 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065289 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-envoy\") pod \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " Jul 13 07:29:56.065646 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065320 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istiod-ca-cert\") pod \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " Jul 13 07:29:56.065646 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065338 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-data\") pod \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " Jul 13 07:29:56.065646 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065491 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95xd2\" (UniqueName: \"kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-kube-api-access-95xd2\") pod \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " Jul 13 07:29:56.065646 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065531 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-podinfo\") pod \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " Jul 13 07:29:56.065646 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065604 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" (UID: "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:29:56.065646 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065618 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-credential-socket\") pod \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " Jul 13 07:29:56.066116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065660 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-socket\") pod \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\" (UID: \"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5\") " Jul 13 07:29:56.066116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065672 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" (UID: "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:29:56.066116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.065936 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" (UID: "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:29:56.066116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.066068 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:56.066116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.066084 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:56.066116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.066085 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" (UID: "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:29:56.066479 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.066096 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:56.066479 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.066197 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" (UID: "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:29:56.067525 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.067500 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" (UID: "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:29:56.067654 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.067637 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" (UID: "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:29:56.067795 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.067777 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" (UID: "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:29:56.067910 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.067893 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-kube-api-access-95xd2" (OuterVolumeSpecName: "kube-api-access-95xd2") pod "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" (UID: "a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5"). InnerVolumeSpecName "kube-api-access-95xd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:29:56.166633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.166612 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:56.166633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.166634 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95xd2\" (UniqueName: \"kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-kube-api-access-95xd2\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:56.166757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.166643 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:56.166757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.166652 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:56.166757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.166659 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:56.166757 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.166667 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:29:56.710115 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.710081 2583 generic.go:356] "Generic (PLEG): container finished" podID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerID="3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897" exitCode=0 Jul 13 07:29:56.710264 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.710131 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" Jul 13 07:29:56.710264 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.710172 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" event={"ID":"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5","Type":"ContainerDied","Data":"3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897"} Jul 13 07:29:56.710264 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.710250 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs" event={"ID":"a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5","Type":"ContainerDied","Data":"c64df9b0f221c0406639171f08b1cde85523f88ce51157457cf847e71fa3c96a"} Jul 13 07:29:56.710417 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.710279 2583 scope.go:117] "RemoveContainer" containerID="3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897" Jul 13 07:29:56.719608 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.719589 2583 scope.go:117] "RemoveContainer" containerID="902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538" Jul 13 07:29:56.729317 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.729289 2583 scope.go:117] "RemoveContainer" containerID="3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897" Jul 13 07:29:56.730222 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:29:56.730175 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897\": container with ID starting with 3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897 not found: ID does not exist" containerID="3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897" Jul 13 07:29:56.730329 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.730232 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897"} err="failed to get container status \"3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897\": rpc error: code = NotFound desc = could not find container \"3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897\": container with ID starting with 3950bfc925ff62cdc390f4f62737781f26ee80a8b2a0cc326b3933afa7dce897 not found: ID does not exist" Jul 13 07:29:56.730329 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.730259 2583 scope.go:117] "RemoveContainer" containerID="902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538" Jul 13 07:29:56.730558 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:29:56.730539 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538\": container with ID starting with 902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538 not found: ID does not exist" containerID="902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538" Jul 13 07:29:56.730618 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.730565 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538"} err="failed to get container status \"902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538\": rpc error: code = NotFound desc = could not find container \"902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538\": container with ID starting with 902aeaf409c1b6da101bf7b71bb388209e4786bbc1104c27728b9d4b813fa538 not found: ID does not exist" Jul 13 07:29:56.732319 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.732300 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs"] Jul 13 07:29:56.737772 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:56.737752 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2d7b36bf-openshift-default-6bf8589d59-bznhs"] Jul 13 07:29:57.429507 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:29:57.429475 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" path="/var/lib/kubelet/pods/a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5/volumes" Jul 13 07:30:01.187793 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:01.187762 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:01.289950 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:01.289917 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:07.951579 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:07.951544 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:08.231402 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.231375 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm"] Jul 13 07:30:08.231852 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.231839 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78" containerName="maas-api" Jul 13 07:30:08.231897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.231855 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78" containerName="maas-api" Jul 13 07:30:08.231897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.231870 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerName="istio-proxy" Jul 13 07:30:08.231897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.231876 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerName="istio-proxy" Jul 13 07:30:08.231897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.231882 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerName="istio-proxy" Jul 13 07:30:08.231897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.231887 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerName="istio-proxy" Jul 13 07:30:08.232046 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.231967 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c67fd7d-56cb-4fa5-b8a2-0bfa2d5fcf78" containerName="maas-api" Jul 13 07:30:08.232046 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.231978 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerName="istio-proxy" Jul 13 07:30:08.232046 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.231985 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5f6ec77-ecf9-40f2-a23a-90efd3a9e4e5" containerName="istio-proxy" Jul 13 07:30:08.235111 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.235093 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.237836 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.237815 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2735700e-openshift-default-dockercfg-2v8tk\"" Jul 13 07:30:08.246380 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.246358 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm"] Jul 13 07:30:08.253859 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.253819 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:08.371892 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.371857 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-credential-socket\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.372053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.371908 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/503eb269-9627-4e17-94b5-51b70a0f3e2d-istiod-ca-cert\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.372053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.371967 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-token\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.372137 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.372050 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-certs\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.372137 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.372088 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t9md\" (UniqueName: \"kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-kube-api-access-6t9md\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.372137 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.372128 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-socket\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.372277 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.372152 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-data\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.372277 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.372175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-envoy\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.372277 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.372261 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-podinfo\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.473592 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.473566 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-token\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.473710 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.473602 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-certs\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.473710 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.473620 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t9md\" (UniqueName: \"kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-kube-api-access-6t9md\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.473710 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.473645 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-socket\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.473710 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.473663 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-data\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.473934 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.473810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-envoy\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.473994 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.473940 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-podinfo\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.474050 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.474010 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-credential-socket\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.474050 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.474018 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-certs\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.474145 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.474053 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/503eb269-9627-4e17-94b5-51b70a0f3e2d-istiod-ca-cert\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.474145 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.474058 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-data\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.474145 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.474090 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-socket\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.474302 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.474282 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-credential-socket\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.474633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.474617 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/503eb269-9627-4e17-94b5-51b70a0f3e2d-istiod-ca-cert\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.475966 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.475946 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-envoy\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.476314 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.476295 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-podinfo\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.483577 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.483531 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-token\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.483698 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.483674 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t9md\" (UniqueName: \"kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-kube-api-access-6t9md\") pod \"e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.537399 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.537375 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:08.545637 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.545614 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:08.680448 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.680425 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm"] Jul 13 07:30:08.682232 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:30:08.682187 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503eb269_9627_4e17_94b5_51b70a0f3e2d.slice/crio-b543eaf2f2f5f015136d4c088486c31fb80d23f046481043713c8bfd0220db4f WatchSource:0}: Error finding container b543eaf2f2f5f015136d4c088486c31fb80d23f046481043713c8bfd0220db4f: Status 404 returned error can't find the container with id b543eaf2f2f5f015136d4c088486c31fb80d23f046481043713c8bfd0220db4f Jul 13 07:30:08.684455 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.684419 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:30:08.684571 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.684505 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:30:08.684571 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.684551 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:30:08.774240 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.774191 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" event={"ID":"503eb269-9627-4e17-94b5-51b70a0f3e2d","Type":"ContainerStarted","Data":"f71898dd966c6e1c5680f4854649604fe1d30c37aa8b361b61a239bb26f944dc"} Jul 13 07:30:08.774466 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.774269 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" event={"ID":"503eb269-9627-4e17-94b5-51b70a0f3e2d","Type":"ContainerStarted","Data":"b543eaf2f2f5f015136d4c088486c31fb80d23f046481043713c8bfd0220db4f"} Jul 13 07:30:08.795760 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:08.795710 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" podStartSLOduration=0.795691834 podStartE2EDuration="795.691834ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:30:08.79523459 +0000 UTC m=+1661.969738414" watchObservedRunningTime="2026-07-13 07:30:08.795691834 +0000 UTC m=+1661.970195640" Jul 13 07:30:09.546170 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:09.546137 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:09.550645 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:09.550622 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:09.779524 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:09.779495 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:09.780141 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:09.780123 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:10.693882 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.687350 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:10.758338 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.758310 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7"] Jul 13 07:30:10.762377 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.762347 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:10.763003 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.762976 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:10.764972 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.764954 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2735700e-serving-cert\"" Jul 13 07:30:10.774168 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.774147 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7"] Jul 13 07:30:10.791627 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.791603 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec5965d-b8af-496a-a30b-49433529fa12-maas-api-tls\") pod \"maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:10.791748 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.791640 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnhcf\" (UniqueName: \"kubernetes.io/projected/cec5965d-b8af-496a-a30b-49433529fa12-kube-api-access-jnhcf\") pod \"maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:10.791860 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.791837 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec5965d-b8af-496a-a30b-49433529fa12-openshift-service-ca\") pod \"maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:10.831175 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.831152 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:10.893059 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.893028 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnhcf\" (UniqueName: \"kubernetes.io/projected/cec5965d-b8af-496a-a30b-49433529fa12-kube-api-access-jnhcf\") pod \"maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:10.893199 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.893141 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec5965d-b8af-496a-a30b-49433529fa12-openshift-service-ca\") pod \"maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:10.893199 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.893189 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec5965d-b8af-496a-a30b-49433529fa12-maas-api-tls\") pod \"maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:10.893328 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:30:10.893313 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-serving-cert: secret "maas-api-e2e-mt-2735700e-serving-cert" not found Jul 13 07:30:10.893413 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:30:10.893403 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cec5965d-b8af-496a-a30b-49433529fa12-maas-api-tls podName:cec5965d-b8af-496a-a30b-49433529fa12 nodeName:}" failed. No retries permitted until 2026-07-13 07:30:11.393378737 +0000 UTC m=+1664.567882542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cec5965d-b8af-496a-a30b-49433529fa12-maas-api-tls") pod "maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" (UID: "cec5965d-b8af-496a-a30b-49433529fa12") : secret "maas-api-e2e-mt-2735700e-serving-cert" not found Jul 13 07:30:10.893878 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.893841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec5965d-b8af-496a-a30b-49433529fa12-openshift-service-ca\") pod \"maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:10.903785 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:10.903759 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnhcf\" (UniqueName: \"kubernetes.io/projected/cec5965d-b8af-496a-a30b-49433529fa12-kube-api-access-jnhcf\") pod \"maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:11.008883 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:11.008857 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:11.245774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:11.245748 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:11.397790 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:11.397709 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec5965d-b8af-496a-a30b-49433529fa12-maas-api-tls\") pod \"maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:11.400129 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:11.400112 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec5965d-b8af-496a-a30b-49433529fa12-maas-api-tls\") pod \"maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:11.438038 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:11.438013 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:11.676282 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:11.676189 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:11.813400 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:11.813374 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7"] Jul 13 07:30:11.814798 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:30:11.814772 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcec5965d_b8af_496a_a30b_49433529fa12.slice/crio-3eb747a423bf61f4d28cb155acb6e9f66918a9daf354fc7b9311b53bb3f6aa03 WatchSource:0}: Error finding container 3eb747a423bf61f4d28cb155acb6e9f66918a9daf354fc7b9311b53bb3f6aa03: Status 404 returned error can't find the container with id 3eb747a423bf61f4d28cb155acb6e9f66918a9daf354fc7b9311b53bb3f6aa03 Jul 13 07:30:12.002917 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:12.002888 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:12.105885 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:12.105846 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:12.797548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:12.797512 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" event={"ID":"cec5965d-b8af-496a-a30b-49433529fa12","Type":"ContainerStarted","Data":"cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b"} Jul 13 07:30:12.797741 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:12.797566 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" event={"ID":"cec5965d-b8af-496a-a30b-49433529fa12","Type":"ContainerStarted","Data":"3eb747a423bf61f4d28cb155acb6e9f66918a9daf354fc7b9311b53bb3f6aa03"} Jul 13 07:30:12.797741 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:12.797638 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:12.815020 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:12.814980 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" podStartSLOduration=2.5338540800000002 podStartE2EDuration="2.814966573s" totalImagesPullingTime="281.112493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:30:12.813481629 +0000 UTC m=+1665.987985435" watchObservedRunningTime="2026-07-13 07:30:12.814966573 +0000 UTC m=+1665.989470380" Jul 13 07:30:15.199364 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:15.199328 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:18.063992 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:18.063966 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:18.808116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:18.808089 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:21.855331 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:21.855280 2583 generic.go:356] "Generic (PLEG): container finished" podID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerID="f71898dd966c6e1c5680f4854649604fe1d30c37aa8b361b61a239bb26f944dc" exitCode=0 Jul 13 07:30:21.855331 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:21.855314 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" event={"ID":"503eb269-9627-4e17-94b5-51b70a0f3e2d","Type":"ContainerDied","Data":"f71898dd966c6e1c5680f4854649604fe1d30c37aa8b361b61a239bb26f944dc"} Jul 13 07:30:21.855678 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:21.855649 2583 scope.go:117] "RemoveContainer" containerID="f71898dd966c6e1c5680f4854649604fe1d30c37aa8b361b61a239bb26f944dc" Jul 13 07:30:21.856600 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:21.856570 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:30:21.856670 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:21.856653 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:30:21.856720 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:21.856706 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:30:22.862631 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:22.862596 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" event={"ID":"503eb269-9627-4e17-94b5-51b70a0f3e2d","Type":"ContainerStarted","Data":"f9192be83becb36de97e203134fd85308a089b4da331a2dd008a161ddec9016c"} Jul 13 07:30:23.546671 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:23.546639 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:23.547791 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:23.547762 2583 patch_prober.go:28] interesting pod/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 13 07:30:23.547938 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:23.547813 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 13 07:30:24.546728 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:24.546691 2583 patch_prober.go:28] interesting pod/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 13 07:30:24.547120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:24.546741 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 13 07:30:24.780071 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:24.780041 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:25.546534 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:25.546499 2583 patch_prober.go:28] interesting pod/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 13 07:30:25.546714 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:25.546553 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 13 07:30:26.550374 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:26.550344 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:26.551302 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:26.551283 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:30:39.656904 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:39.656867 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:39.758624 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:39.758589 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:47.729496 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:47.729464 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:48.245145 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.245100 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:48.343379 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.343354 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:48.467641 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.467610 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7"] Jul 13 07:30:48.467847 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.467820 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" podUID="cec5965d-b8af-496a-a30b-49433529fa12" containerName="maas-api" containerID="cri-o://cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b" gracePeriod=30 Jul 13 07:30:48.490040 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.490014 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:48.716873 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.716851 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:48.838123 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.838054 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec5965d-b8af-496a-a30b-49433529fa12-maas-api-tls\") pod \"cec5965d-b8af-496a-a30b-49433529fa12\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " Jul 13 07:30:48.838123 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.838089 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec5965d-b8af-496a-a30b-49433529fa12-openshift-service-ca\") pod \"cec5965d-b8af-496a-a30b-49433529fa12\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " Jul 13 07:30:48.838123 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.838121 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnhcf\" (UniqueName: \"kubernetes.io/projected/cec5965d-b8af-496a-a30b-49433529fa12-kube-api-access-jnhcf\") pod \"cec5965d-b8af-496a-a30b-49433529fa12\" (UID: \"cec5965d-b8af-496a-a30b-49433529fa12\") " Jul 13 07:30:48.838528 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.838458 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cec5965d-b8af-496a-a30b-49433529fa12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cec5965d-b8af-496a-a30b-49433529fa12" (UID: "cec5965d-b8af-496a-a30b-49433529fa12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:30:48.839905 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.839882 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cec5965d-b8af-496a-a30b-49433529fa12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cec5965d-b8af-496a-a30b-49433529fa12" (UID: "cec5965d-b8af-496a-a30b-49433529fa12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:30:48.840014 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.839994 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cec5965d-b8af-496a-a30b-49433529fa12-kube-api-access-jnhcf" (OuterVolumeSpecName: "kube-api-access-jnhcf") pod "cec5965d-b8af-496a-a30b-49433529fa12" (UID: "cec5965d-b8af-496a-a30b-49433529fa12"). InnerVolumeSpecName "kube-api-access-jnhcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:30:48.939218 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.939181 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec5965d-b8af-496a-a30b-49433529fa12-maas-api-tls\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:30:48.939326 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.939235 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec5965d-b8af-496a-a30b-49433529fa12-openshift-service-ca\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:30:48.939326 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.939252 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnhcf\" (UniqueName: \"kubernetes.io/projected/cec5965d-b8af-496a-a30b-49433529fa12-kube-api-access-jnhcf\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:30:48.995158 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.995133 2583 generic.go:356] "Generic (PLEG): container finished" podID="cec5965d-b8af-496a-a30b-49433529fa12" containerID="cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b" exitCode=0 Jul 13 07:30:48.995287 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.995179 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" Jul 13 07:30:48.995287 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.995240 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" event={"ID":"cec5965d-b8af-496a-a30b-49433529fa12","Type":"ContainerDied","Data":"cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b"} Jul 13 07:30:48.995287 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.995285 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7" event={"ID":"cec5965d-b8af-496a-a30b-49433529fa12","Type":"ContainerDied","Data":"3eb747a423bf61f4d28cb155acb6e9f66918a9daf354fc7b9311b53bb3f6aa03"} Jul 13 07:30:48.995476 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:48.995307 2583 scope.go:117] "RemoveContainer" containerID="cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b" Jul 13 07:30:49.004426 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:49.004406 2583 scope.go:117] "RemoveContainer" containerID="cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b" Jul 13 07:30:49.004692 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:30:49.004674 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b\": container with ID starting with cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b not found: ID does not exist" containerID="cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b" Jul 13 07:30:49.004736 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:49.004701 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b"} err="failed to get container status \"cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b\": rpc error: code = NotFound desc = could not find container \"cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b\": container with ID starting with cddf0d50854373399a735dec931759715d8f4f88d24e5bf21e0d56751761ca9b not found: ID does not exist" Jul 13 07:30:49.016761 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:49.016714 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7"] Jul 13 07:30:49.020441 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:49.020421 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2735700e-6db47cb7f8-bsdt7"] Jul 13 07:30:49.428431 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:49.428402 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cec5965d-b8af-496a-a30b-49433529fa12" path="/var/lib/kubelet/pods/cec5965d-b8af-496a-a30b-49433529fa12/volumes" Jul 13 07:30:53.836486 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:53.836456 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:30:58.883414 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:30:58.883370 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:31:34.962629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:31:34.962591 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:31:36.022335 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:31:36.022295 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:31:58.908852 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:31:58.908800 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:31:59.009399 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:31:59.009370 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:31:59.189278 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:31:59.189186 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm"] Jul 13 07:31:59.189435 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:31:59.189414 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerName="istio-proxy" containerID="cri-o://f9192be83becb36de97e203134fd85308a089b4da331a2dd008a161ddec9016c" gracePeriod=30 Jul 13 07:31:59.200411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:31:59.200382 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:00.502696 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:00.502662 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:00.602249 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:00.602219 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:00.713117 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:00.713092 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:01.455198 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:01.455156 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:01.552051 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:01.552008 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:01.652507 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:01.652477 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:03.045990 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:03.045936 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:04.385089 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.385059 2583 generic.go:356] "Generic (PLEG): container finished" podID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerID="f9192be83becb36de97e203134fd85308a089b4da331a2dd008a161ddec9016c" exitCode=0 Jul 13 07:32:04.385495 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.385256 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" event={"ID":"503eb269-9627-4e17-94b5-51b70a0f3e2d","Type":"ContainerDied","Data":"f9192be83becb36de97e203134fd85308a089b4da331a2dd008a161ddec9016c"} Jul 13 07:32:04.385495 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.385328 2583 scope.go:117] "RemoveContainer" containerID="f71898dd966c6e1c5680f4854649604fe1d30c37aa8b361b61a239bb26f944dc" Jul 13 07:32:04.458963 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.458938 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:32:04.531069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531032 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-credential-socket\") pod \"503eb269-9627-4e17-94b5-51b70a0f3e2d\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " Jul 13 07:32:04.531253 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531081 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/503eb269-9627-4e17-94b5-51b70a0f3e2d-istiod-ca-cert\") pod \"503eb269-9627-4e17-94b5-51b70a0f3e2d\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " Jul 13 07:32:04.531253 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531161 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-token\") pod \"503eb269-9627-4e17-94b5-51b70a0f3e2d\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " Jul 13 07:32:04.531253 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531233 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-certs\") pod \"503eb269-9627-4e17-94b5-51b70a0f3e2d\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " Jul 13 07:32:04.531388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531282 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-envoy\") pod \"503eb269-9627-4e17-94b5-51b70a0f3e2d\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " Jul 13 07:32:04.531388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531313 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-socket\") pod \"503eb269-9627-4e17-94b5-51b70a0f3e2d\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " Jul 13 07:32:04.531388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531327 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "503eb269-9627-4e17-94b5-51b70a0f3e2d" (UID: "503eb269-9627-4e17-94b5-51b70a0f3e2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:32:04.531388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531346 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-data\") pod \"503eb269-9627-4e17-94b5-51b70a0f3e2d\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " Jul 13 07:32:04.531596 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531425 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-podinfo\") pod \"503eb269-9627-4e17-94b5-51b70a0f3e2d\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " Jul 13 07:32:04.531596 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531470 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t9md\" (UniqueName: \"kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-kube-api-access-6t9md\") pod \"503eb269-9627-4e17-94b5-51b70a0f3e2d\" (UID: \"503eb269-9627-4e17-94b5-51b70a0f3e2d\") " Jul 13 07:32:04.531596 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531499 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503eb269-9627-4e17-94b5-51b70a0f3e2d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "503eb269-9627-4e17-94b5-51b70a0f3e2d" (UID: "503eb269-9627-4e17-94b5-51b70a0f3e2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:32:04.531767 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531612 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "503eb269-9627-4e17-94b5-51b70a0f3e2d" (UID: "503eb269-9627-4e17-94b5-51b70a0f3e2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:32:04.531767 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531608 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "503eb269-9627-4e17-94b5-51b70a0f3e2d" (UID: "503eb269-9627-4e17-94b5-51b70a0f3e2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:32:04.532004 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.531977 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:32:04.532130 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.532006 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/503eb269-9627-4e17-94b5-51b70a0f3e2d-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:32:04.532130 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.532024 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:32:04.532130 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.532046 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:32:04.532580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.532538 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-data" (OuterVolumeSpecName: "istio-data") pod "503eb269-9627-4e17-94b5-51b70a0f3e2d" (UID: "503eb269-9627-4e17-94b5-51b70a0f3e2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:32:04.533706 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.533677 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "503eb269-9627-4e17-94b5-51b70a0f3e2d" (UID: "503eb269-9627-4e17-94b5-51b70a0f3e2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:32:04.533806 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.533712 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "503eb269-9627-4e17-94b5-51b70a0f3e2d" (UID: "503eb269-9627-4e17-94b5-51b70a0f3e2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:32:04.533806 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.533787 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-token" (OuterVolumeSpecName: "istio-token") pod "503eb269-9627-4e17-94b5-51b70a0f3e2d" (UID: "503eb269-9627-4e17-94b5-51b70a0f3e2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:32:04.534146 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.534122 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-kube-api-access-6t9md" (OuterVolumeSpecName: "kube-api-access-6t9md") pod "503eb269-9627-4e17-94b5-51b70a0f3e2d" (UID: "503eb269-9627-4e17-94b5-51b70a0f3e2d"). InnerVolumeSpecName "kube-api-access-6t9md". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:32:04.633151 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.633128 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:32:04.633151 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.633148 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:32:04.633302 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.633159 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:32:04.633302 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.633170 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/503eb269-9627-4e17-94b5-51b70a0f3e2d-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:32:04.633302 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:04.633183 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t9md\" (UniqueName: \"kubernetes.io/projected/503eb269-9627-4e17-94b5-51b70a0f3e2d-kube-api-access-6t9md\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:32:05.400284 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:05.400241 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" event={"ID":"503eb269-9627-4e17-94b5-51b70a0f3e2d","Type":"ContainerDied","Data":"b543eaf2f2f5f015136d4c088486c31fb80d23f046481043713c8bfd0220db4f"} Jul 13 07:32:05.400284 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:05.400275 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm" Jul 13 07:32:05.400828 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:05.400328 2583 scope.go:117] "RemoveContainer" containerID="f9192be83becb36de97e203134fd85308a089b4da331a2dd008a161ddec9016c" Jul 13 07:32:05.428326 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:05.428302 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm"] Jul 13 07:32:05.433442 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:05.433417 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2735700e-openshift-default-68d968ddd7-64jfm"] Jul 13 07:32:07.428389 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:07.428362 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" path="/var/lib/kubelet/pods/503eb269-9627-4e17-94b5-51b70a0f3e2d/volumes" Jul 13 07:32:08.955967 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:08.955939 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:09.060177 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:09.060149 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:09.653250 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:09.653191 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:15.534459 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:15.534419 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:15.637650 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:15.637617 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:15.735986 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:15.735948 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:26.316732 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:26.316697 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:26.648347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:26.648274 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:47.688086 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:47.688047 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:54.082586 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.082551 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:54.181615 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.181580 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:54.457014 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.456934 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:54.834376 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.834340 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f"] Jul 13 07:32:54.834856 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.834837 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerName="istio-proxy" Jul 13 07:32:54.834952 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.834857 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerName="istio-proxy" Jul 13 07:32:54.834952 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.834872 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerName="istio-proxy" Jul 13 07:32:54.834952 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.834879 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerName="istio-proxy" Jul 13 07:32:54.834952 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.834909 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cec5965d-b8af-496a-a30b-49433529fa12" containerName="maas-api" Jul 13 07:32:54.834952 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.834918 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cec5965d-b8af-496a-a30b-49433529fa12" containerName="maas-api" Jul 13 07:32:54.835217 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.835028 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerName="istio-proxy" Jul 13 07:32:54.835217 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.835041 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="503eb269-9627-4e17-94b5-51b70a0f3e2d" containerName="istio-proxy" Jul 13 07:32:54.835217 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.835052 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cec5965d-b8af-496a-a30b-49433529fa12" containerName="maas-api" Jul 13 07:32:54.838431 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.838408 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:54.843238 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.843219 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8ccb17-openshift-default-dockercfg-wgg95\"" Jul 13 07:32:54.860179 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.860156 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f"] Jul 13 07:32:54.862931 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.862897 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:54.957085 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.957054 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-envoy\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:54.957257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.957096 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-data\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:54.957257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.957115 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4916e534-2671-475d-bff5-b4edc828e8bb-istio-podinfo\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:54.957257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.957213 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-istio-token\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:54.957371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.957268 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-credential-socket\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:54.957371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.957289 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-certs\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:54.957371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.957314 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-socket\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:54.957371 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.957331 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dql5\" (UniqueName: \"kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-kube-api-access-5dql5\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:54.957496 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.957395 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4916e534-2671-475d-bff5-b4edc828e8bb-istiod-ca-cert\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:54.992998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:54.992965 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:55.058667 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.058638 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-envoy\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.058815 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.058684 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-data\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.058815 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.058708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4916e534-2671-475d-bff5-b4edc828e8bb-istio-podinfo\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.058815 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.058757 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-istio-token\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.058815 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.058799 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-credential-socket\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.059031 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.058818 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-certs\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.059031 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.058851 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-socket\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.059031 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.058889 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dql5\" (UniqueName: \"kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-kube-api-access-5dql5\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.059031 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.058923 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4916e534-2671-475d-bff5-b4edc828e8bb-istiod-ca-cert\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.059293 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.059059 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-data\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.059293 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.059190 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-certs\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.059293 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.059272 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-credential-socket\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.059407 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.059373 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-socket\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.059580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.059560 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4916e534-2671-475d-bff5-b4edc828e8bb-istiod-ca-cert\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.061462 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.061444 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-envoy\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.061786 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.061766 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4916e534-2671-475d-bff5-b4edc828e8bb-istio-podinfo\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.072897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.072873 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dql5\" (UniqueName: \"kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-kube-api-access-5dql5\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.073337 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.073321 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-istio-token\") pod \"e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.149175 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.149126 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:55.301998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.301967 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f"] Jul 13 07:32:55.302490 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:32:55.302465 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4916e534_2671_475d_bff5_b4edc828e8bb.slice/crio-fc917941ac89822e476d7eb7525c52a46d430ca05f31cc33c38201526ed13091 WatchSource:0}: Error finding container fc917941ac89822e476d7eb7525c52a46d430ca05f31cc33c38201526ed13091: Status 404 returned error can't find the container with id fc917941ac89822e476d7eb7525c52a46d430ca05f31cc33c38201526ed13091 Jul 13 07:32:55.304387 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.304358 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:32:55.304438 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.304418 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:32:55.304478 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.304448 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:32:55.654662 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.654630 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" event={"ID":"4916e534-2671-475d-bff5-b4edc828e8bb","Type":"ContainerStarted","Data":"bc869e0cb4b3ab78d57116f1be277610ba883c99cf151155fc85fa6501783f95"} Jul 13 07:32:55.654819 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.654679 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" event={"ID":"4916e534-2671-475d-bff5-b4edc828e8bb","Type":"ContainerStarted","Data":"fc917941ac89822e476d7eb7525c52a46d430ca05f31cc33c38201526ed13091"} Jul 13 07:32:55.682583 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:55.682529 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" podStartSLOduration=1.682513938 podStartE2EDuration="1.682513938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:32:55.681336556 +0000 UTC m=+1828.855840364" watchObservedRunningTime="2026-07-13 07:32:55.682513938 +0000 UTC m=+1828.857017744" Jul 13 07:32:56.150283 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:56.150254 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:56.154970 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:56.154946 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:56.659850 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:56.659820 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:56.660679 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:56.660662 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:32:57.936510 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:57.936477 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:58.134511 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:58.134483 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:58.296169 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:58.296137 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:58.880008 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:58.879977 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:59.189339 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:59.189244 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:32:59.331338 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:32:59.331305 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:02.288539 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:02.288504 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:02.609901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:02.609798 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:02.901682 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:02.901613 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq"] Jul 13 07:33:02.905799 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:02.905783 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:02.911771 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:02.911746 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-3d88d9-openshift-default-dockercfg-2bgbl\"" Jul 13 07:33:02.923113 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:02.923091 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq"] Jul 13 07:33:02.944963 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:02.944922 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:03.035251 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.035192 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-credential-socket\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.035416 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.035282 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-socket\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.035416 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.035374 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-podinfo\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.035416 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.035390 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa13f17-db58-4e2b-b979-38b23ec828ab-istiod-ca-cert\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.035530 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.035427 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-envoy\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.035530 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.035442 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xchd9\" (UniqueName: \"kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-kube-api-access-xchd9\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.035530 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.035475 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-data\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.035530 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.035498 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-token\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.035530 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.035517 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-certs\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.136633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.136606 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-podinfo\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.136633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.136635 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa13f17-db58-4e2b-b979-38b23ec828ab-istiod-ca-cert\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.136808 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.136657 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-envoy\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.136808 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.136672 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xchd9\" (UniqueName: \"kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-kube-api-access-xchd9\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.136808 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.136695 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-data\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.136940 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.136815 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-token\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.136940 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.136857 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-certs\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.136940 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.136926 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-credential-socket\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.137089 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.136975 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-socket\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.137089 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.137069 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-data\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.137249 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.137229 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-certs\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.137424 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.137399 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-credential-socket\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.137424 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.137417 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-socket\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.137557 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.137509 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa13f17-db58-4e2b-b979-38b23ec828ab-istiod-ca-cert\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.139031 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.139007 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-envoy\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.139159 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.139143 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-podinfo\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.149144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.149123 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-token\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.151035 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.151012 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xchd9\" (UniqueName: \"kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-kube-api-access-xchd9\") pod \"e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.217310 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.217289 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:03.235914 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.235887 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:03.363021 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.362992 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq"] Jul 13 07:33:03.364168 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:33:03.364146 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa13f17_db58_4e2b_b979_38b23ec828ab.slice/crio-3c68276ac16de3388115dc579065e5087d1828b3d73263b081554717080429b2 WatchSource:0}: Error finding container 3c68276ac16de3388115dc579065e5087d1828b3d73263b081554717080429b2: Status 404 returned error can't find the container with id 3c68276ac16de3388115dc579065e5087d1828b3d73263b081554717080429b2 Jul 13 07:33:03.366306 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.366273 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:33:03.366412 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.366348 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:33:03.366412 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.366393 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:33:03.701596 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.701514 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" event={"ID":"6aa13f17-db58-4e2b-b979-38b23ec828ab","Type":"ContainerStarted","Data":"d6cc20e5e742dee1e6d8e35195ea383533c1192f2372fc3dc6d32fd0196f8ce7"} Jul 13 07:33:03.701596 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.701563 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" event={"ID":"6aa13f17-db58-4e2b-b979-38b23ec828ab","Type":"ContainerStarted","Data":"3c68276ac16de3388115dc579065e5087d1828b3d73263b081554717080429b2"} Jul 13 07:33:03.730563 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:03.730520 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" podStartSLOduration=1.730504125 podStartE2EDuration="1.730504125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:33:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:33:03.727689062 +0000 UTC m=+1836.902192866" watchObservedRunningTime="2026-07-13 07:33:03.730504125 +0000 UTC m=+1836.905007932" Jul 13 07:33:04.217703 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:04.217666 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:04.222317 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:04.222293 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:04.706472 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:04.706396 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:04.707186 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:04.707164 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:05.690794 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.690755 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:05.759176 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.759142 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22"] Jul 13 07:33:05.762945 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.762922 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:05.766129 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.766107 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3d88d9-serving-cert\"" Jul 13 07:33:05.812698 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.812674 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:05.840006 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.839984 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22"] Jul 13 07:33:05.867627 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.867593 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121f2239-fc1a-4672-8413-ae8a9a6c6f40-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:05.867796 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.867749 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jws45\" (UniqueName: \"kubernetes.io/projected/121f2239-fc1a-4672-8413-ae8a9a6c6f40-kube-api-access-jws45\") pod \"maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:05.867859 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.867827 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121f2239-fc1a-4672-8413-ae8a9a6c6f40-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:05.915886 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.915856 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:05.969383 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.969357 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121f2239-fc1a-4672-8413-ae8a9a6c6f40-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:05.969522 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.969480 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jws45\" (UniqueName: \"kubernetes.io/projected/121f2239-fc1a-4672-8413-ae8a9a6c6f40-kube-api-access-jws45\") pod \"maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:05.969594 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.969551 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121f2239-fc1a-4672-8413-ae8a9a6c6f40-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:05.969652 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:33:05.969635 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-serving-cert: secret "maas-api-e2e-inf-b-3d88d9-serving-cert" not found Jul 13 07:33:05.969702 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:33:05.969694 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/121f2239-fc1a-4672-8413-ae8a9a6c6f40-maas-api-tls podName:121f2239-fc1a-4672-8413-ae8a9a6c6f40 nodeName:}" failed. No retries permitted until 2026-07-13 07:33:06.469678601 +0000 UTC m=+1839.644182385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/121f2239-fc1a-4672-8413-ae8a9a6c6f40-maas-api-tls") pod "maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" (UID: "121f2239-fc1a-4672-8413-ae8a9a6c6f40") : secret "maas-api-e2e-inf-b-3d88d9-serving-cert" not found Jul 13 07:33:05.969885 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.969866 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121f2239-fc1a-4672-8413-ae8a9a6c6f40-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:05.979189 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:05.979170 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jws45\" (UniqueName: \"kubernetes.io/projected/121f2239-fc1a-4672-8413-ae8a9a6c6f40-kube-api-access-jws45\") pod \"maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:06.038387 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:06.038362 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:06.393677 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:06.393591 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:06.473937 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:06.473909 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121f2239-fc1a-4672-8413-ae8a9a6c6f40-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:06.476356 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:06.476336 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121f2239-fc1a-4672-8413-ae8a9a6c6f40-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:06.549461 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:06.549428 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:06.673880 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:06.673790 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:06.817575 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:06.817546 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22"] Jul 13 07:33:06.819149 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:33:06.819111 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121f2239_fc1a_4672_8413_ae8a9a6c6f40.slice/crio-bb9f870d9f60d5279cf342f2c9ecca9de6f13ce49ad3f2f528d683bb953a0b2e WatchSource:0}: Error finding container bb9f870d9f60d5279cf342f2c9ecca9de6f13ce49ad3f2f528d683bb953a0b2e: Status 404 returned error can't find the container with id bb9f870d9f60d5279cf342f2c9ecca9de6f13ce49ad3f2f528d683bb953a0b2e Jul 13 07:33:06.821308 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:06.821190 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:33:07.724058 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:07.724023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" event={"ID":"121f2239-fc1a-4672-8413-ae8a9a6c6f40","Type":"ContainerStarted","Data":"f9f8062c424946cec88d306cc0b517a7caad6711305ae456e4a387c56bc5c9da"} Jul 13 07:33:07.724271 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:07.724074 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" event={"ID":"121f2239-fc1a-4672-8413-ae8a9a6c6f40","Type":"ContainerStarted","Data":"bb9f870d9f60d5279cf342f2c9ecca9de6f13ce49ad3f2f528d683bb953a0b2e"} Jul 13 07:33:07.724271 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:07.724178 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:07.743523 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:07.743476 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" podStartSLOduration=2.417843506 podStartE2EDuration="2.743462445s" totalImagesPullingTime="325.618939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:33:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:33:07.741726033 +0000 UTC m=+1840.916229841" watchObservedRunningTime="2026-07-13 07:33:07.743462445 +0000 UTC m=+1840.917966252" Jul 13 07:33:10.217395 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:10.217365 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:12.701426 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:12.701385 2583 patch_prober.go:28] interesting pod/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 07:33:12.702070 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:12.702031 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 07:33:13.735918 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:13.735891 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:33:14.764809 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:14.764776 2583 generic.go:356] "Generic (PLEG): container finished" podID="4916e534-2671-475d-bff5-b4edc828e8bb" containerID="bc869e0cb4b3ab78d57116f1be277610ba883c99cf151155fc85fa6501783f95" exitCode=0 Jul 13 07:33:14.765256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:14.764827 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" event={"ID":"4916e534-2671-475d-bff5-b4edc828e8bb","Type":"ContainerDied","Data":"bc869e0cb4b3ab78d57116f1be277610ba883c99cf151155fc85fa6501783f95"} Jul 13 07:33:14.765256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:14.765162 2583 scope.go:117] "RemoveContainer" containerID="bc869e0cb4b3ab78d57116f1be277610ba883c99cf151155fc85fa6501783f95" Jul 13 07:33:14.766116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:14.766091 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:33:14.766177 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:14.766159 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:33:14.766240 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:14.766193 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:33:15.771309 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:15.771276 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" event={"ID":"4916e534-2671-475d-bff5-b4edc828e8bb","Type":"ContainerStarted","Data":"7ee787100f53160027e4bf69247e1002826ab84220f2a9692c790f5a9d393fe0"} Jul 13 07:33:16.149621 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:16.149519 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:33:16.149799 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:16.149666 2583 patch_prober.go:28] interesting pod/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 13 07:33:16.149799 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:16.149727 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 13 07:33:17.149816 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:17.149763 2583 patch_prober.go:28] interesting pod/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 13 07:33:17.150240 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:17.149841 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 13 07:33:18.150376 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:18.150339 2583 patch_prober.go:28] interesting pod/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 13 07:33:18.150735 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:18.150393 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 13 07:33:19.149880 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:19.149842 2583 patch_prober.go:28] interesting pod/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 13 07:33:19.150061 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:19.149902 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 13 07:33:20.153505 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:20.153476 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:33:20.153986 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:20.153862 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:33:20.154452 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:20.154432 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:33:20.747348 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:20.747314 2583 patch_prober.go:28] interesting pod/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 07:33:20.747525 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:20.747375 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 07:33:21.279776 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:21.279748 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:21.377727 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:21.377695 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:21.828623 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:21.828590 2583 generic.go:356] "Generic (PLEG): container finished" podID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerID="d6cc20e5e742dee1e6d8e35195ea383533c1192f2372fc3dc6d32fd0196f8ce7" exitCode=0 Jul 13 07:33:21.828808 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:21.828656 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" event={"ID":"6aa13f17-db58-4e2b-b979-38b23ec828ab","Type":"ContainerDied","Data":"d6cc20e5e742dee1e6d8e35195ea383533c1192f2372fc3dc6d32fd0196f8ce7"} Jul 13 07:33:21.829192 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:21.829174 2583 scope.go:117] "RemoveContainer" containerID="d6cc20e5e742dee1e6d8e35195ea383533c1192f2372fc3dc6d32fd0196f8ce7" Jul 13 07:33:21.830194 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:21.830171 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:33:21.830285 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:21.830252 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:33:21.830332 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:21.830288 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:33:22.837048 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:22.837000 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" event={"ID":"6aa13f17-db58-4e2b-b979-38b23ec828ab","Type":"ContainerStarted","Data":"b1a9b902ba8cd6b72cf5638523d297f02b4299c443e05562e5eeee955a78e555"} Jul 13 07:33:23.218450 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:23.218419 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:23.218620 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:23.218596 2583 patch_prober.go:28] interesting pod/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 07:33:23.218664 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:23.218648 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 07:33:24.217663 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:24.217625 2583 patch_prober.go:28] interesting pod/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 07:33:24.218061 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:24.217677 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 07:33:25.218487 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:25.218448 2583 patch_prober.go:28] interesting pod/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 07:33:25.218957 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:25.218514 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 07:33:26.217967 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:26.217926 2583 patch_prober.go:28] interesting pod/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 07:33:26.218140 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:26.217974 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 07:33:27.220900 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:27.220871 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:27.221360 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:27.221331 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:27.222039 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:27.222022 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:33:27.349582 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:27.349543 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:27.451011 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:27.450978 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:33:28.348422 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:33:28.348388 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:34:14.682447 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:34:14.682372 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:34:44.572360 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:34:44.572332 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:34:45.486938 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:34:45.486903 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:34:45.775980 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:34:45.775896 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:34:45.955436 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:34:45.955409 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:34:46.149050 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:34:46.148980 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:34:46.304575 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:34:46.304541 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:34:46.407385 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:34:46.407326 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:34:46.562496 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:34:46.562467 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:34:51.310493 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:34:51.310457 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:21.167189 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:21.167155 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:21.226598 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:21.226568 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:28.057919 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:28.057887 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:28.309737 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:28.309652 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f"] Jul 13 07:35:28.309932 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:28.309886 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" containerID="cri-o://7ee787100f53160027e4bf69247e1002826ab84220f2a9692c790f5a9d393fe0" gracePeriod=30 Jul 13 07:35:28.333613 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:28.333585 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:28.709886 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:28.709834 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:29.125133 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.125066 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:29.226057 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.226033 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:29.339585 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.339545 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22"] Jul 13 07:35:29.339836 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.339806 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" podUID="121f2239-fc1a-4672-8413-ae8a9a6c6f40" containerName="maas-api" containerID="cri-o://f9f8062c424946cec88d306cc0b517a7caad6711305ae456e4a387c56bc5c9da" gracePeriod=30 Jul 13 07:35:29.378621 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.378519 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:29.472688 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.472659 2583 generic.go:356] "Generic (PLEG): container finished" podID="121f2239-fc1a-4672-8413-ae8a9a6c6f40" containerID="f9f8062c424946cec88d306cc0b517a7caad6711305ae456e4a387c56bc5c9da" exitCode=0 Jul 13 07:35:29.472830 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.472713 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" event={"ID":"121f2239-fc1a-4672-8413-ae8a9a6c6f40","Type":"ContainerDied","Data":"f9f8062c424946cec88d306cc0b517a7caad6711305ae456e4a387c56bc5c9da"} Jul 13 07:35:29.593375 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.593352 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:35:29.748183 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.748159 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jws45\" (UniqueName: \"kubernetes.io/projected/121f2239-fc1a-4672-8413-ae8a9a6c6f40-kube-api-access-jws45\") pod \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " Jul 13 07:35:29.748342 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.748303 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121f2239-fc1a-4672-8413-ae8a9a6c6f40-maas-api-tls\") pod \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " Jul 13 07:35:29.748408 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.748354 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121f2239-fc1a-4672-8413-ae8a9a6c6f40-openshift-service-ca\") pod \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\" (UID: \"121f2239-fc1a-4672-8413-ae8a9a6c6f40\") " Jul 13 07:35:29.748728 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.748707 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/121f2239-fc1a-4672-8413-ae8a9a6c6f40-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "121f2239-fc1a-4672-8413-ae8a9a6c6f40" (UID: "121f2239-fc1a-4672-8413-ae8a9a6c6f40"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:35:29.750100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.750079 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/121f2239-fc1a-4672-8413-ae8a9a6c6f40-kube-api-access-jws45" (OuterVolumeSpecName: "kube-api-access-jws45") pod "121f2239-fc1a-4672-8413-ae8a9a6c6f40" (UID: "121f2239-fc1a-4672-8413-ae8a9a6c6f40"). InnerVolumeSpecName "kube-api-access-jws45". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:35:29.750259 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.750234 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/121f2239-fc1a-4672-8413-ae8a9a6c6f40-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "121f2239-fc1a-4672-8413-ae8a9a6c6f40" (UID: "121f2239-fc1a-4672-8413-ae8a9a6c6f40"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:35:29.849872 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.849831 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121f2239-fc1a-4672-8413-ae8a9a6c6f40-maas-api-tls\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:29.849974 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.849875 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121f2239-fc1a-4672-8413-ae8a9a6c6f40-openshift-service-ca\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:29.849974 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:29.849889 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jws45\" (UniqueName: \"kubernetes.io/projected/121f2239-fc1a-4672-8413-ae8a9a6c6f40-kube-api-access-jws45\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:30.478877 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:30.478852 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" Jul 13 07:35:30.478877 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:30.478866 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22" event={"ID":"121f2239-fc1a-4672-8413-ae8a9a6c6f40","Type":"ContainerDied","Data":"bb9f870d9f60d5279cf342f2c9ecca9de6f13ce49ad3f2f528d683bb953a0b2e"} Jul 13 07:35:30.479390 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:30.478929 2583 scope.go:117] "RemoveContainer" containerID="f9f8062c424946cec88d306cc0b517a7caad6711305ae456e4a387c56bc5c9da" Jul 13 07:35:30.501282 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:30.501259 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22"] Jul 13 07:35:30.505220 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:30.505192 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d88d9-74f8cf4dc6-r8x22"] Jul 13 07:35:31.428978 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:31.428948 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="121f2239-fc1a-4672-8413-ae8a9a6c6f40" path="/var/lib/kubelet/pods/121f2239-fc1a-4672-8413-ae8a9a6c6f40/volumes" Jul 13 07:35:33.497514 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.497485 2583 generic.go:356] "Generic (PLEG): container finished" podID="4916e534-2671-475d-bff5-b4edc828e8bb" containerID="7ee787100f53160027e4bf69247e1002826ab84220f2a9692c790f5a9d393fe0" exitCode=0 Jul 13 07:35:33.497947 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.497564 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" event={"ID":"4916e534-2671-475d-bff5-b4edc828e8bb","Type":"ContainerDied","Data":"7ee787100f53160027e4bf69247e1002826ab84220f2a9692c790f5a9d393fe0"} Jul 13 07:35:33.497947 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.497642 2583 scope.go:117] "RemoveContainer" containerID="bc869e0cb4b3ab78d57116f1be277610ba883c99cf151155fc85fa6501783f95" Jul 13 07:35:33.561465 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.561446 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:35:33.683702 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.683639 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-credential-socket\") pod \"4916e534-2671-475d-bff5-b4edc828e8bb\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " Jul 13 07:35:33.683819 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.683706 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-istio-token\") pod \"4916e534-2671-475d-bff5-b4edc828e8bb\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " Jul 13 07:35:33.683819 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.683746 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4916e534-2671-475d-bff5-b4edc828e8bb-istiod-ca-cert\") pod \"4916e534-2671-475d-bff5-b4edc828e8bb\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " Jul 13 07:35:33.683819 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.683779 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-socket\") pod \"4916e534-2671-475d-bff5-b4edc828e8bb\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " Jul 13 07:35:33.683819 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.683812 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dql5\" (UniqueName: \"kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-kube-api-access-5dql5\") pod \"4916e534-2671-475d-bff5-b4edc828e8bb\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " Jul 13 07:35:33.684034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.683852 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-envoy\") pod \"4916e534-2671-475d-bff5-b4edc828e8bb\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " Jul 13 07:35:33.684034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.683901 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4916e534-2671-475d-bff5-b4edc828e8bb-istio-podinfo\") pod \"4916e534-2671-475d-bff5-b4edc828e8bb\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " Jul 13 07:35:33.684034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.683903 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4916e534-2671-475d-bff5-b4edc828e8bb" (UID: "4916e534-2671-475d-bff5-b4edc828e8bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:33.684034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.683935 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-data\") pod \"4916e534-2671-475d-bff5-b4edc828e8bb\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " Jul 13 07:35:33.684034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.683985 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-certs\") pod \"4916e534-2671-475d-bff5-b4edc828e8bb\" (UID: \"4916e534-2671-475d-bff5-b4edc828e8bb\") " Jul 13 07:35:33.684034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.684011 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4916e534-2671-475d-bff5-b4edc828e8bb" (UID: "4916e534-2671-475d-bff5-b4edc828e8bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:33.684363 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.684096 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4916e534-2671-475d-bff5-b4edc828e8bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4916e534-2671-475d-bff5-b4edc828e8bb" (UID: "4916e534-2671-475d-bff5-b4edc828e8bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:35:33.684363 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.684326 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4916e534-2671-475d-bff5-b4edc828e8bb" (UID: "4916e534-2671-475d-bff5-b4edc828e8bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:33.684482 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.684464 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4916e534-2671-475d-bff5-b4edc828e8bb-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:33.684547 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.684488 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:33.684547 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.684501 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:33.684547 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.684517 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:33.684975 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.684950 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "4916e534-2671-475d-bff5-b4edc828e8bb" (UID: "4916e534-2671-475d-bff5-b4edc828e8bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:33.685885 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.685839 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "4916e534-2671-475d-bff5-b4edc828e8bb" (UID: "4916e534-2671-475d-bff5-b4edc828e8bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:35:33.685985 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.685950 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-kube-api-access-5dql5" (OuterVolumeSpecName: "kube-api-access-5dql5") pod "4916e534-2671-475d-bff5-b4edc828e8bb" (UID: "4916e534-2671-475d-bff5-b4edc828e8bb"). InnerVolumeSpecName "kube-api-access-5dql5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:35:33.686298 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.686277 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4916e534-2671-475d-bff5-b4edc828e8bb" (UID: "4916e534-2671-475d-bff5-b4edc828e8bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:33.686349 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.686319 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4916e534-2671-475d-bff5-b4edc828e8bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4916e534-2671-475d-bff5-b4edc828e8bb" (UID: "4916e534-2671-475d-bff5-b4edc828e8bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:35:33.785899 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.785875 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:33.785899 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.785896 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dql5\" (UniqueName: \"kubernetes.io/projected/4916e534-2671-475d-bff5-b4edc828e8bb-kube-api-access-5dql5\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:33.786033 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.785910 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:33.786033 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.785920 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4916e534-2671-475d-bff5-b4edc828e8bb-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:33.786033 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:33.785927 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4916e534-2671-475d-bff5-b4edc828e8bb-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:34.504054 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:34.504030 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" Jul 13 07:35:34.504499 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:34.504061 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f" event={"ID":"4916e534-2671-475d-bff5-b4edc828e8bb","Type":"ContainerDied","Data":"fc917941ac89822e476d7eb7525c52a46d430ca05f31cc33c38201526ed13091"} Jul 13 07:35:34.504499 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:34.504105 2583 scope.go:117] "RemoveContainer" containerID="7ee787100f53160027e4bf69247e1002826ab84220f2a9692c790f5a9d393fe0" Jul 13 07:35:34.526167 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:34.526142 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f"] Jul 13 07:35:34.530316 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:34.530294 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8ccb17-openshift-default-6fbf58cc89-7875f"] Jul 13 07:35:35.051551 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:35.051522 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:35.151198 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:35.151167 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:35.307264 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:35.307182 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq"] Jul 13 07:35:35.307408 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:35.307384 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" containerID="cri-o://b1a9b902ba8cd6b72cf5638523d297f02b4299c443e05562e5eeee955a78e555" gracePeriod=30 Jul 13 07:35:35.330728 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:35.330699 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:35.428961 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:35.428929 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" path="/var/lib/kubelet/pods/4916e534-2671-475d-bff5-b4edc828e8bb/volumes" Jul 13 07:35:35.886729 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:35.886701 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:36.175327 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.175251 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g"] Jul 13 07:35:36.175871 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.175845 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" Jul 13 07:35:36.175871 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.175868 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" Jul 13 07:35:36.176064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.175893 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" Jul 13 07:35:36.176064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.175900 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" Jul 13 07:35:36.176064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.175918 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="121f2239-fc1a-4672-8413-ae8a9a6c6f40" containerName="maas-api" Jul 13 07:35:36.176064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.175927 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="121f2239-fc1a-4672-8413-ae8a9a6c6f40" containerName="maas-api" Jul 13 07:35:36.176064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.176044 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="121f2239-fc1a-4672-8413-ae8a9a6c6f40" containerName="maas-api" Jul 13 07:35:36.176064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.176058 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" Jul 13 07:35:36.176436 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.176071 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4916e534-2671-475d-bff5-b4edc828e8bb" containerName="istio-proxy" Jul 13 07:35:36.180287 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.180269 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.182677 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.182656 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-1f88a7-openshift-default-dockercfg-tqttr\"" Jul 13 07:35:36.192988 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.192955 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g"] Jul 13 07:35:36.202573 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.202555 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:36.308901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.308869 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-credential-socket\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.308901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.308903 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-certs\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.309120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.308942 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-token\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.309120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.308964 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-podinfo\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.309120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.308985 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-data\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.309120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.309002 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-envoy\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.309120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.309031 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-socket\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.309120 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.309088 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d54q\" (UniqueName: \"kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-kube-api-access-4d54q\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.309355 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.309155 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istiod-ca-cert\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.409802 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.409777 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istiod-ca-cert\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.409955 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.409887 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-credential-socket\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.409955 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.409915 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-certs\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.409962 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-token\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.409987 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-podinfo\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.410012 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-data\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410064 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.410041 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-envoy\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410301 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.410200 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-socket\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410301 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.410281 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d54q\" (UniqueName: \"kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-kube-api-access-4d54q\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410409 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.410380 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-credential-socket\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410466 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.410424 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-data\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410626 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.410593 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-socket\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410725 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.410702 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-certs\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.410776 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.410738 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istiod-ca-cert\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.412402 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.412383 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-envoy\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.412516 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.412502 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-podinfo\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.419858 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.419838 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-token\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.420155 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.420134 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d54q\" (UniqueName: \"kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-kube-api-access-4d54q\") pod \"e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.490752 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.490729 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:36.543739 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.543266 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:36.631699 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.631675 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g"] Jul 13 07:35:36.633368 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:35:36.633341 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod082bce6b_c329_4ba8_bacc_7f71a05cd6a0.slice/crio-d270c04e30d5506f36fe0517ff06847d6904f874f8af5eb7acdde3c6ef661fc3 WatchSource:0}: Error finding container d270c04e30d5506f36fe0517ff06847d6904f874f8af5eb7acdde3c6ef661fc3: Status 404 returned error can't find the container with id d270c04e30d5506f36fe0517ff06847d6904f874f8af5eb7acdde3c6ef661fc3 Jul 13 07:35:36.635459 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.635432 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:36.635545 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.635488 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:36.635545 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:36.635515 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:37.522271 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:37.522239 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" event={"ID":"082bce6b-c329-4ba8-bacc-7f71a05cd6a0","Type":"ContainerStarted","Data":"3bac88e1bfc3eef0201c7e9805166ab8e807b4fbef1b2845b780c968b50ba263"} Jul 13 07:35:37.522624 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:37.522286 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" event={"ID":"082bce6b-c329-4ba8-bacc-7f71a05cd6a0","Type":"ContainerStarted","Data":"d270c04e30d5506f36fe0517ff06847d6904f874f8af5eb7acdde3c6ef661fc3"} Jul 13 07:35:37.546903 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:37.546860 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" podStartSLOduration=1.546843701 podStartE2EDuration="1.546843701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:35:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:35:37.544111738 +0000 UTC m=+1990.718615592" watchObservedRunningTime="2026-07-13 07:35:37.546843701 +0000 UTC m=+1990.721347569" Jul 13 07:35:38.491269 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:38.491238 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:38.495833 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:38.495809 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:38.527445 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:38.527424 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:38.528090 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:38.528069 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:39.472055 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:39.472024 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:39.587531 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:39.587484 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:39.665875 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:39.665825 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:39.804032 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:39.803961 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:40.142522 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.142417 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:40.262000 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.261971 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:40.543566 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.543541 2583 generic.go:356] "Generic (PLEG): container finished" podID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerID="b1a9b902ba8cd6b72cf5638523d297f02b4299c443e05562e5eeee955a78e555" exitCode=0 Jul 13 07:35:40.543703 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.543615 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" event={"ID":"6aa13f17-db58-4e2b-b979-38b23ec828ab","Type":"ContainerDied","Data":"b1a9b902ba8cd6b72cf5638523d297f02b4299c443e05562e5eeee955a78e555"} Jul 13 07:35:40.543703 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.543682 2583 scope.go:117] "RemoveContainer" containerID="d6cc20e5e742dee1e6d8e35195ea383533c1192f2372fc3dc6d32fd0196f8ce7" Jul 13 07:35:40.575618 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.575600 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:35:40.644893 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.644872 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-socket\") pod \"6aa13f17-db58-4e2b-b979-38b23ec828ab\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " Jul 13 07:35:40.645244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.644904 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-podinfo\") pod \"6aa13f17-db58-4e2b-b979-38b23ec828ab\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " Jul 13 07:35:40.645244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.644928 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-certs\") pod \"6aa13f17-db58-4e2b-b979-38b23ec828ab\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " Jul 13 07:35:40.645244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.644956 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-data\") pod \"6aa13f17-db58-4e2b-b979-38b23ec828ab\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " Jul 13 07:35:40.645244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.644973 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-token\") pod \"6aa13f17-db58-4e2b-b979-38b23ec828ab\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " Jul 13 07:35:40.645244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.644992 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa13f17-db58-4e2b-b979-38b23ec828ab-istiod-ca-cert\") pod \"6aa13f17-db58-4e2b-b979-38b23ec828ab\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " Jul 13 07:35:40.645244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.645023 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-credential-socket\") pod \"6aa13f17-db58-4e2b-b979-38b23ec828ab\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " Jul 13 07:35:40.645244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.645051 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-envoy\") pod \"6aa13f17-db58-4e2b-b979-38b23ec828ab\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " Jul 13 07:35:40.645244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.645094 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xchd9\" (UniqueName: \"kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-kube-api-access-xchd9\") pod \"6aa13f17-db58-4e2b-b979-38b23ec828ab\" (UID: \"6aa13f17-db58-4e2b-b979-38b23ec828ab\") " Jul 13 07:35:40.645244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.645140 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6aa13f17-db58-4e2b-b979-38b23ec828ab" (UID: "6aa13f17-db58-4e2b-b979-38b23ec828ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:40.645723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.645305 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6aa13f17-db58-4e2b-b979-38b23ec828ab" (UID: "6aa13f17-db58-4e2b-b979-38b23ec828ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:40.645723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.645394 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aa13f17-db58-4e2b-b979-38b23ec828ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6aa13f17-db58-4e2b-b979-38b23ec828ab" (UID: "6aa13f17-db58-4e2b-b979-38b23ec828ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:35:40.645723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.645547 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6aa13f17-db58-4e2b-b979-38b23ec828ab" (UID: "6aa13f17-db58-4e2b-b979-38b23ec828ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:40.645723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.645559 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:40.645723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.645577 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:40.645723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.645592 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa13f17-db58-4e2b-b979-38b23ec828ab-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:40.647101 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.647036 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "6aa13f17-db58-4e2b-b979-38b23ec828ab" (UID: "6aa13f17-db58-4e2b-b979-38b23ec828ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:40.647623 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.647597 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6aa13f17-db58-4e2b-b979-38b23ec828ab" (UID: "6aa13f17-db58-4e2b-b979-38b23ec828ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:35:40.647718 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.647649 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "6aa13f17-db58-4e2b-b979-38b23ec828ab" (UID: "6aa13f17-db58-4e2b-b979-38b23ec828ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:35:40.647718 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.647661 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-kube-api-access-xchd9" (OuterVolumeSpecName: "kube-api-access-xchd9") pod "6aa13f17-db58-4e2b-b979-38b23ec828ab" (UID: "6aa13f17-db58-4e2b-b979-38b23ec828ab"). InnerVolumeSpecName "kube-api-access-xchd9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:35:40.655514 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.655482 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6aa13f17-db58-4e2b-b979-38b23ec828ab" (UID: "6aa13f17-db58-4e2b-b979-38b23ec828ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:40.746799 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.746779 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:40.746799 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.746799 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:40.746930 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.746807 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:40.746930 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.746816 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:40.746930 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.746824 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa13f17-db58-4e2b-b979-38b23ec828ab-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:40.746930 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:40.746849 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xchd9\" (UniqueName: \"kubernetes.io/projected/6aa13f17-db58-4e2b-b979-38b23ec828ab-kube-api-access-xchd9\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:35:41.186101 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:41.185986 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:41.287116 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:41.287087 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:41.396969 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:41.396937 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:41.550709 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:41.550679 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" event={"ID":"6aa13f17-db58-4e2b-b979-38b23ec828ab","Type":"ContainerDied","Data":"3c68276ac16de3388115dc579065e5087d1828b3d73263b081554717080429b2"} Jul 13 07:35:41.550849 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:41.550721 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq" Jul 13 07:35:41.550849 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:41.550731 2583 scope.go:117] "RemoveContainer" containerID="b1a9b902ba8cd6b72cf5638523d297f02b4299c443e05562e5eeee955a78e555" Jul 13 07:35:41.586556 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:41.586527 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq"] Jul 13 07:35:41.593348 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:41.593329 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-3d88d9-openshift-default-6bf547b4f9-p9mnq"] Jul 13 07:35:42.047921 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:42.047888 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:42.145113 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:42.145085 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:43.429037 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:43.429006 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" path="/var/lib/kubelet/pods/6aa13f17-db58-4e2b-b979-38b23ec828ab/volumes" Jul 13 07:35:44.003231 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.003170 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:44.477034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.477013 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:44.759189 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.759120 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn"] Jul 13 07:35:44.759569 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.759555 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" Jul 13 07:35:44.759611 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.759570 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" Jul 13 07:35:44.759611 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.759580 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" Jul 13 07:35:44.759611 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.759585 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" Jul 13 07:35:44.759703 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.759644 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" Jul 13 07:35:44.759703 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.759652 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aa13f17-db58-4e2b-b979-38b23ec828ab" containerName="istio-proxy" Jul 13 07:35:44.762714 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.762698 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.765399 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.765379 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-780e06-openshift-default-dockercfg-c799z\"" Jul 13 07:35:44.772695 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.772675 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn"] Jul 13 07:35:44.781611 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.781589 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vw68\" (UniqueName: \"kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-kube-api-access-8vw68\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.781721 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.781626 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-data\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.781721 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.781647 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b13dd2a1-1c28-4544-ae21-653522a2ec41-istiod-ca-cert\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.781721 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.781667 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-podinfo\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.781721 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.781711 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-socket\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.781911 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.781758 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-credential-socket\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.781911 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.781776 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-certs\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.781911 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.781789 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-envoy\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.781911 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.781881 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-token\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.783461 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.783441 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:44.882430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882408 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-data\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.882523 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882436 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b13dd2a1-1c28-4544-ae21-653522a2ec41-istiod-ca-cert\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.882523 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882457 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-podinfo\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.882602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882576 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-socket\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.882664 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882647 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-credential-socket\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.882723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882681 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-certs\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.882723 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-envoy\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.882836 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882760 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-token\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.882836 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882789 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-data\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.882949 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882855 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vw68\" (UniqueName: \"kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-kube-api-access-8vw68\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.882949 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882918 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-socket\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.883076 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.882991 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-credential-socket\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.883188 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.883170 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-certs\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.883563 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.883342 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b13dd2a1-1c28-4544-ae21-653522a2ec41-istiod-ca-cert\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.887226 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.885974 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-envoy\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.887226 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.885985 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-podinfo\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.891486 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.891462 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-token\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.892640 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.892618 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vw68\" (UniqueName: \"kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-kube-api-access-8vw68\") pod \"e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:44.951209 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:44.951187 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:45.073509 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:45.073428 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:45.211944 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:45.211916 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn"] Jul 13 07:35:45.213675 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:35:45.213652 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13dd2a1_1c28_4544_ae21_653522a2ec41.slice/crio-835d4c23bb6d4a399829053d0eccfde87ff1a0f02599a5d58c1b49f84ea2d40f WatchSource:0}: Error finding container 835d4c23bb6d4a399829053d0eccfde87ff1a0f02599a5d58c1b49f84ea2d40f: Status 404 returned error can't find the container with id 835d4c23bb6d4a399829053d0eccfde87ff1a0f02599a5d58c1b49f84ea2d40f Jul 13 07:35:45.215782 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:45.215750 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:45.215861 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:45.215810 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:45.215861 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:45.215843 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:45.576397 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:45.576363 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" event={"ID":"b13dd2a1-1c28-4544-ae21-653522a2ec41","Type":"ContainerStarted","Data":"2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521"} Jul 13 07:35:45.576761 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:45.576423 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" event={"ID":"b13dd2a1-1c28-4544-ae21-653522a2ec41","Type":"ContainerStarted","Data":"835d4c23bb6d4a399829053d0eccfde87ff1a0f02599a5d58c1b49f84ea2d40f"} Jul 13 07:35:45.598939 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:45.598894 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" podStartSLOduration=1.5988777220000001 podStartE2EDuration="1.598877722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:35:45.597499379 +0000 UTC m=+1998.772003186" watchObservedRunningTime="2026-07-13 07:35:45.598877722 +0000 UTC m=+1998.773381530" Jul 13 07:35:46.073774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:46.073742 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:46.078588 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:46.078565 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:46.582294 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:46.582260 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:46.584597 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:46.584371 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:35:47.497980 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.497948 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:47.578396 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.578359 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk"] Jul 13 07:35:47.581129 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.581108 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:47.591300 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.587417 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:47.591300 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.591180 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk"] Jul 13 07:35:47.591739 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.591393 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-780e06-serving-cert\"" Jul 13 07:35:47.618234 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.616729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v84x8\" (UniqueName: \"kubernetes.io/projected/58b264f9-4c03-4cc5-84db-8f006a9ee48a-kube-api-access-v84x8\") pod \"maas-api-e2e-api-b-780e06-84c6b4885-rkwfk\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:47.618234 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.616800 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b264f9-4c03-4cc5-84db-8f006a9ee48a-openshift-service-ca\") pod \"maas-api-e2e-api-b-780e06-84c6b4885-rkwfk\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:47.618234 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.616877 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b264f9-4c03-4cc5-84db-8f006a9ee48a-maas-api-tls\") pod \"maas-api-e2e-api-b-780e06-84c6b4885-rkwfk\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:47.669380 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.669351 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:47.718232 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.717484 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b264f9-4c03-4cc5-84db-8f006a9ee48a-openshift-service-ca\") pod \"maas-api-e2e-api-b-780e06-84c6b4885-rkwfk\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:47.718232 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.717619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b264f9-4c03-4cc5-84db-8f006a9ee48a-maas-api-tls\") pod \"maas-api-e2e-api-b-780e06-84c6b4885-rkwfk\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:47.718232 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.717718 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v84x8\" (UniqueName: \"kubernetes.io/projected/58b264f9-4c03-4cc5-84db-8f006a9ee48a-kube-api-access-v84x8\") pod \"maas-api-e2e-api-b-780e06-84c6b4885-rkwfk\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:47.718559 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:35:47.718282 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-serving-cert: secret "maas-api-e2e-api-b-780e06-serving-cert" not found Jul 13 07:35:47.718559 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:35:47.718354 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58b264f9-4c03-4cc5-84db-8f006a9ee48a-maas-api-tls podName:58b264f9-4c03-4cc5-84db-8f006a9ee48a nodeName:}" failed. No retries permitted until 2026-07-13 07:35:48.218339595 +0000 UTC m=+2001.392843380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/58b264f9-4c03-4cc5-84db-8f006a9ee48a-maas-api-tls") pod "maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" (UID: "58b264f9-4c03-4cc5-84db-8f006a9ee48a") : secret "maas-api-e2e-api-b-780e06-serving-cert" not found Jul 13 07:35:47.719101 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.719078 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b264f9-4c03-4cc5-84db-8f006a9ee48a-openshift-service-ca\") pod \"maas-api-e2e-api-b-780e06-84c6b4885-rkwfk\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:47.730015 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:47.729995 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v84x8\" (UniqueName: \"kubernetes.io/projected/58b264f9-4c03-4cc5-84db-8f006a9ee48a-kube-api-access-v84x8\") pod \"maas-api-e2e-api-b-780e06-84c6b4885-rkwfk\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:48.031149 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:48.031113 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:48.221774 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:48.221742 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b264f9-4c03-4cc5-84db-8f006a9ee48a-maas-api-tls\") pod \"maas-api-e2e-api-b-780e06-84c6b4885-rkwfk\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:48.224220 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:48.224182 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b264f9-4c03-4cc5-84db-8f006a9ee48a-maas-api-tls\") pod \"maas-api-e2e-api-b-780e06-84c6b4885-rkwfk\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:48.238311 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:48.238285 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:48.339112 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:48.339023 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:48.521647 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:48.521606 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:48.711397 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:48.711372 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk"] Jul 13 07:35:49.628450 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:49.628412 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" event={"ID":"58b264f9-4c03-4cc5-84db-8f006a9ee48a","Type":"ContainerStarted","Data":"bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2"} Jul 13 07:35:49.628622 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:49.628485 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" event={"ID":"58b264f9-4c03-4cc5-84db-8f006a9ee48a","Type":"ContainerStarted","Data":"3ea2426544d8378483b19552bceb04db1b2fa8e5c431c656e1f97e9cd555a097"} Jul 13 07:35:49.628622 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:49.628534 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:49.648404 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:49.648362 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" podStartSLOduration=2.327769794 podStartE2EDuration="2.64834743s" totalImagesPullingTime="320.577636ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:35:49.645325337 +0000 UTC m=+2002.819829142" watchObservedRunningTime="2026-07-13 07:35:49.64834743 +0000 UTC m=+2002.822851235" Jul 13 07:35:51.996798 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:51.996715 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:35:53.528769 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:53.528739 2583 patch_prober.go:28] interesting pod/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 13 07:35:53.529149 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:53.528789 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 13 07:35:53.653256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:53.653227 2583 generic.go:356] "Generic (PLEG): container finished" podID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerID="3bac88e1bfc3eef0201c7e9805166ab8e807b4fbef1b2845b780c968b50ba263" exitCode=0 Jul 13 07:35:53.653401 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:53.653303 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" event={"ID":"082bce6b-c329-4ba8-bacc-7f71a05cd6a0","Type":"ContainerDied","Data":"3bac88e1bfc3eef0201c7e9805166ab8e807b4fbef1b2845b780c968b50ba263"} Jul 13 07:35:53.653739 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:53.653723 2583 scope.go:117] "RemoveContainer" containerID="3bac88e1bfc3eef0201c7e9805166ab8e807b4fbef1b2845b780c968b50ba263" Jul 13 07:35:53.654736 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:53.654705 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:53.654856 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:53.654793 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:53.654856 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:53.654840 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:54.660597 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:54.660567 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" event={"ID":"082bce6b-c329-4ba8-bacc-7f71a05cd6a0","Type":"ContainerStarted","Data":"278ca2c2e13af8fcc1d20acdf90916073cfd28829d97a171fc26e3792086e0c7"} Jul 13 07:35:55.491493 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:55.491450 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:55.491758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:55.491731 2583 patch_prober.go:28] interesting pod/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 13 07:35:55.491856 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:55.491788 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 13 07:35:55.641722 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:55.641694 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:35:56.492601 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:56.492156 2583 patch_prober.go:28] interesting pod/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 13 07:35:56.492601 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:56.492238 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 13 07:35:57.491247 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:57.491193 2583 patch_prober.go:28] interesting pod/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 13 07:35:57.491537 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:57.491506 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 13 07:35:58.491517 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:58.491482 2583 patch_prober.go:28] interesting pod/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 13 07:35:58.492089 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:58.492059 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 13 07:35:59.495404 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:59.495379 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:59.495722 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:59.495636 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:59.496348 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:59.496332 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:35:59.705656 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:59.705583 2583 generic.go:356] "Generic (PLEG): container finished" podID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerID="2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521" exitCode=0 Jul 13 07:35:59.705789 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:59.705655 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" event={"ID":"b13dd2a1-1c28-4544-ae21-653522a2ec41","Type":"ContainerDied","Data":"2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521"} Jul 13 07:35:59.706175 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:59.706162 2583 scope.go:117] "RemoveContainer" containerID="2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521" Jul 13 07:35:59.707100 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:59.707074 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:59.707168 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:59.707152 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:35:59.707232 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:35:59.707215 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:00.044744 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:00.044714 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:00.495095 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:00.495062 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:00.682390 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:00.682332 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:00.714028 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:00.713993 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" event={"ID":"b13dd2a1-1c28-4544-ae21-653522a2ec41","Type":"ContainerStarted","Data":"bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e"} Jul 13 07:36:00.786882 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:00.786800 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:00.950547 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:00.950510 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:01.074374 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:01.074290 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:36:01.075543 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:01.075509 2583 patch_prober.go:28] interesting pod/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 13 07:36:01.075703 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:01.075556 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 13 07:36:01.582547 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:01.582505 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:36:02.074585 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:02.074551 2583 patch_prober.go:28] interesting pod/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 13 07:36:02.074957 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:02.074602 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 13 07:36:03.074044 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:03.074007 2583 patch_prober.go:28] interesting pod/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 13 07:36:03.074242 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:03.074059 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 13 07:36:04.074480 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:04.074446 2583 patch_prober.go:28] interesting pod/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 13 07:36:04.074850 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:04.074496 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 13 07:36:05.074739 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:05.074702 2583 patch_prober.go:28] interesting pod/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 13 07:36:05.075187 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:05.074763 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 13 07:36:06.077185 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:06.077157 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:36:06.084944 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:06.084917 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:36:06.299870 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:06.299829 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:06.366440 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:06.366374 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:06.552787 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:06.552755 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g"] Jul 13 07:36:06.553023 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:06.552997 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" containerID="cri-o://278ca2c2e13af8fcc1d20acdf90916073cfd28829d97a171fc26e3792086e0c7" gracePeriod=30 Jul 13 07:36:06.579195 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:06.579165 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:07.010464 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:07.010430 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:07.357033 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:07.356964 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:07.462668 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:07.462642 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:07.558839 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:07.558817 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:07.683670 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:07.683591 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk"] Jul 13 07:36:07.683816 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:07.683789 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" podUID="58b264f9-4c03-4cc5-84db-8f006a9ee48a" containerName="maas-api" containerID="cri-o://bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2" gracePeriod=30 Jul 13 07:36:07.718490 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:07.718444 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:07.943480 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:07.943426 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:36:08.028498 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.028473 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b264f9-4c03-4cc5-84db-8f006a9ee48a-maas-api-tls\") pod \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " Jul 13 07:36:08.028618 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.028517 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v84x8\" (UniqueName: \"kubernetes.io/projected/58b264f9-4c03-4cc5-84db-8f006a9ee48a-kube-api-access-v84x8\") pod \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " Jul 13 07:36:08.028698 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.028683 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b264f9-4c03-4cc5-84db-8f006a9ee48a-openshift-service-ca\") pod \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\" (UID: \"58b264f9-4c03-4cc5-84db-8f006a9ee48a\") " Jul 13 07:36:08.028991 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.028968 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58b264f9-4c03-4cc5-84db-8f006a9ee48a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58b264f9-4c03-4cc5-84db-8f006a9ee48a" (UID: "58b264f9-4c03-4cc5-84db-8f006a9ee48a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:36:08.030597 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.030573 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b264f9-4c03-4cc5-84db-8f006a9ee48a-kube-api-access-v84x8" (OuterVolumeSpecName: "kube-api-access-v84x8") pod "58b264f9-4c03-4cc5-84db-8f006a9ee48a" (UID: "58b264f9-4c03-4cc5-84db-8f006a9ee48a"). InnerVolumeSpecName "kube-api-access-v84x8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:36:08.030822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.030797 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58b264f9-4c03-4cc5-84db-8f006a9ee48a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58b264f9-4c03-4cc5-84db-8f006a9ee48a" (UID: "58b264f9-4c03-4cc5-84db-8f006a9ee48a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:36:08.130016 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.129992 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b264f9-4c03-4cc5-84db-8f006a9ee48a-openshift-service-ca\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:08.130016 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.130014 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b264f9-4c03-4cc5-84db-8f006a9ee48a-maas-api-tls\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:08.130143 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.130023 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v84x8\" (UniqueName: \"kubernetes.io/projected/58b264f9-4c03-4cc5-84db-8f006a9ee48a-kube-api-access-v84x8\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:08.758901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.758871 2583 generic.go:356] "Generic (PLEG): container finished" podID="58b264f9-4c03-4cc5-84db-8f006a9ee48a" containerID="bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2" exitCode=0 Jul 13 07:36:08.759388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.758914 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" event={"ID":"58b264f9-4c03-4cc5-84db-8f006a9ee48a","Type":"ContainerDied","Data":"bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2"} Jul 13 07:36:08.759388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.758924 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" Jul 13 07:36:08.759388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.758972 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk" event={"ID":"58b264f9-4c03-4cc5-84db-8f006a9ee48a","Type":"ContainerDied","Data":"3ea2426544d8378483b19552bceb04db1b2fa8e5c431c656e1f97e9cd555a097"} Jul 13 07:36:08.759388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.759005 2583 scope.go:117] "RemoveContainer" containerID="bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2" Jul 13 07:36:08.768547 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.768532 2583 scope.go:117] "RemoveContainer" containerID="bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2" Jul 13 07:36:08.768828 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:36:08.768805 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2\": container with ID starting with bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2 not found: ID does not exist" containerID="bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2" Jul 13 07:36:08.768880 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.768836 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2"} err="failed to get container status \"bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2\": rpc error: code = NotFound desc = could not find container \"bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2\": container with ID starting with bec24ba2e7cfecf16b6e433362527b279dbe0c46240d93aa5e94afd297ed96d2 not found: ID does not exist" Jul 13 07:36:08.782700 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.782672 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk"] Jul 13 07:36:08.790197 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:08.790176 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-780e06-84c6b4885-rkwfk"] Jul 13 07:36:09.428490 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:09.428461 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58b264f9-4c03-4cc5-84db-8f006a9ee48a" path="/var/lib/kubelet/pods/58b264f9-4c03-4cc5-84db-8f006a9ee48a/volumes" Jul 13 07:36:11.778307 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.778275 2583 generic.go:356] "Generic (PLEG): container finished" podID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerID="278ca2c2e13af8fcc1d20acdf90916073cfd28829d97a171fc26e3792086e0c7" exitCode=0 Jul 13 07:36:11.778758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.778345 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" event={"ID":"082bce6b-c329-4ba8-bacc-7f71a05cd6a0","Type":"ContainerDied","Data":"278ca2c2e13af8fcc1d20acdf90916073cfd28829d97a171fc26e3792086e0c7"} Jul 13 07:36:11.778758 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.778435 2583 scope.go:117] "RemoveContainer" containerID="3bac88e1bfc3eef0201c7e9805166ab8e807b4fbef1b2845b780c968b50ba263" Jul 13 07:36:11.813272 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.813247 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:36:11.961990 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.961970 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-credential-socket\") pod \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " Jul 13 07:36:11.962108 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.961997 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-envoy\") pod \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " Jul 13 07:36:11.962108 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962021 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-podinfo\") pod \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " Jul 13 07:36:11.962108 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962069 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d54q\" (UniqueName: \"kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-kube-api-access-4d54q\") pod \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " Jul 13 07:36:11.962291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962110 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istiod-ca-cert\") pod \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " Jul 13 07:36:11.962291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962196 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-socket\") pod \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " Jul 13 07:36:11.962291 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962195 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "082bce6b-c329-4ba8-bacc-7f71a05cd6a0" (UID: "082bce6b-c329-4ba8-bacc-7f71a05cd6a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:36:11.962457 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962393 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-token\") pod \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " Jul 13 07:36:11.962457 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962435 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-data\") pod \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " Jul 13 07:36:11.962550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962455 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "082bce6b-c329-4ba8-bacc-7f71a05cd6a0" (UID: "082bce6b-c329-4ba8-bacc-7f71a05cd6a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:36:11.962550 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962462 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-certs\") pod \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\" (UID: \"082bce6b-c329-4ba8-bacc-7f71a05cd6a0\") " Jul 13 07:36:11.962658 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962566 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "082bce6b-c329-4ba8-bacc-7f71a05cd6a0" (UID: "082bce6b-c329-4ba8-bacc-7f71a05cd6a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:36:11.962658 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962642 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "082bce6b-c329-4ba8-bacc-7f71a05cd6a0" (UID: "082bce6b-c329-4ba8-bacc-7f71a05cd6a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:36:11.962875 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962852 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:11.962875 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962876 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:11.963089 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962890 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:11.963089 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.962904 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:11.963277 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.963254 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "082bce6b-c329-4ba8-bacc-7f71a05cd6a0" (UID: "082bce6b-c329-4ba8-bacc-7f71a05cd6a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:36:11.964321 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.964295 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "082bce6b-c329-4ba8-bacc-7f71a05cd6a0" (UID: "082bce6b-c329-4ba8-bacc-7f71a05cd6a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:36:11.964414 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.964397 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-kube-api-access-4d54q" (OuterVolumeSpecName: "kube-api-access-4d54q") pod "082bce6b-c329-4ba8-bacc-7f71a05cd6a0" (UID: "082bce6b-c329-4ba8-bacc-7f71a05cd6a0"). InnerVolumeSpecName "kube-api-access-4d54q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:36:11.964460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.964442 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "082bce6b-c329-4ba8-bacc-7f71a05cd6a0" (UID: "082bce6b-c329-4ba8-bacc-7f71a05cd6a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:36:11.964497 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:11.964461 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "082bce6b-c329-4ba8-bacc-7f71a05cd6a0" (UID: "082bce6b-c329-4ba8-bacc-7f71a05cd6a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:36:12.064053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:12.064031 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:12.064053 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:12.064050 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:12.064177 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:12.064060 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:12.064177 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:12.064068 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:12.064177 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:12.064076 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d54q\" (UniqueName: \"kubernetes.io/projected/082bce6b-c329-4ba8-bacc-7f71a05cd6a0-kube-api-access-4d54q\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:12.784826 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:12.784788 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" event={"ID":"082bce6b-c329-4ba8-bacc-7f71a05cd6a0","Type":"ContainerDied","Data":"d270c04e30d5506f36fe0517ff06847d6904f874f8af5eb7acdde3c6ef661fc3"} Jul 13 07:36:12.784826 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:12.784813 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g" Jul 13 07:36:12.785347 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:12.784850 2583 scope.go:117] "RemoveContainer" containerID="278ca2c2e13af8fcc1d20acdf90916073cfd28829d97a171fc26e3792086e0c7" Jul 13 07:36:12.807016 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:12.806993 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g"] Jul 13 07:36:12.812243 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:12.812220 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-1f88a7-openshift-default-6c7d67d565-nxt2g"] Jul 13 07:36:13.235637 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:13.235603 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:13.306188 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:13.306149 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:13.429541 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:13.429515 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" path="/var/lib/kubelet/pods/082bce6b-c329-4ba8-bacc-7f71a05cd6a0/volumes" Jul 13 07:36:13.486776 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:13.486706 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn"] Jul 13 07:36:13.486952 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:13.486929 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" containerID="cri-o://bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e" gracePeriod=30 Jul 13 07:36:13.514059 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:13.514034 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:14.069112 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.069083 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:14.369893 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.369795 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf"] Jul 13 07:36:14.370263 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.370250 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58b264f9-4c03-4cc5-84db-8f006a9ee48a" containerName="maas-api" Jul 13 07:36:14.370320 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.370265 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="58b264f9-4c03-4cc5-84db-8f006a9ee48a" containerName="maas-api" Jul 13 07:36:14.370320 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.370293 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" Jul 13 07:36:14.370320 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.370299 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" Jul 13 07:36:14.370320 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.370312 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" Jul 13 07:36:14.370320 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.370319 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" Jul 13 07:36:14.370492 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.370380 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="58b264f9-4c03-4cc5-84db-8f006a9ee48a" containerName="maas-api" Jul 13 07:36:14.370492 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.370390 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" Jul 13 07:36:14.370492 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.370398 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="082bce6b-c329-4ba8-bacc-7f71a05cd6a0" containerName="istio-proxy" Jul 13 07:36:14.373274 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.373254 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.377176 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.377156 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-cc53e7-openshift-default-dockercfg-7n2sj\"" Jul 13 07:36:14.387243 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.387222 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-podinfo\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.387328 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.387273 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-credential-socket\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.387328 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.387306 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-envoy\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.387419 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.387338 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-data\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.387419 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.387356 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istiod-ca-cert\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.387419 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.387385 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-token\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.387419 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.387402 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-socket\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.387570 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.387459 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-certs\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.387570 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.387502 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh6br\" (UniqueName: \"kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-kube-api-access-qh6br\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.392854 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.392833 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf"] Jul 13 07:36:14.399980 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.399944 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:14.488654 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.488625 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-envoy\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.488822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.488661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-data\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.488822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.488680 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istiod-ca-cert\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.488822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.488704 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-token\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.488822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.488733 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-socket\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.488822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.488766 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-certs\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.488822 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.488793 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh6br\" (UniqueName: \"kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-kube-api-access-qh6br\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.489126 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.488874 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-podinfo\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.489126 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.488955 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-credential-socket\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.489126 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.489112 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-data\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.489327 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.489229 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-certs\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.489467 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.489442 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istiod-ca-cert\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.489600 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.489578 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-credential-socket\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.489600 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.489597 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-socket\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.490989 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.490970 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-envoy\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.491404 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.491385 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-podinfo\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.497185 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.497163 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-token\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.497697 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.497677 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh6br\" (UniqueName: \"kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-kube-api-access-qh6br\") pod \"e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.683389 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.683333 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:14.702177 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.702151 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:14.822557 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.822530 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf"] Jul 13 07:36:14.824154 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:36:14.824124 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda674627c_1fc4_48ac_bb29_e9d07c84a3d3.slice/crio-6d19ca57301d9d5f884cb477409414bcd7e6e13fddf698a2292a01c6629376dc WatchSource:0}: Error finding container 6d19ca57301d9d5f884cb477409414bcd7e6e13fddf698a2292a01c6629376dc: Status 404 returned error can't find the container with id 6d19ca57301d9d5f884cb477409414bcd7e6e13fddf698a2292a01c6629376dc Jul 13 07:36:14.826410 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.826382 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:14.826502 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.826442 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:14.826502 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:14.826469 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:15.804096 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:15.804061 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" event={"ID":"a674627c-1fc4-48ac-bb29-e9d07c84a3d3","Type":"ContainerStarted","Data":"c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084"} Jul 13 07:36:15.804545 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:15.804118 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" event={"ID":"a674627c-1fc4-48ac-bb29-e9d07c84a3d3","Type":"ContainerStarted","Data":"6d19ca57301d9d5f884cb477409414bcd7e6e13fddf698a2292a01c6629376dc"} Jul 13 07:36:15.824809 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:15.824760 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" podStartSLOduration=1.824742776 podStartE2EDuration="1.824742776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:36:15.824042575 +0000 UTC m=+2028.998546384" watchObservedRunningTime="2026-07-13 07:36:15.824742776 +0000 UTC m=+2028.999246656" Jul 13 07:36:16.683914 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:16.683884 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:16.688461 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:16.688437 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:16.809341 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:16.809318 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:16.810257 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:16.810240 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:17.778093 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:17.778030 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:17.883644 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:17.883618 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:17.983568 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:17.983536 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:18.087996 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.087911 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:18.513377 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.513352 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:18.615016 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.614969 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:18.729928 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.729905 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:36:18.822096 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.822012 2583 generic.go:356] "Generic (PLEG): container finished" podID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerID="bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e" exitCode=0 Jul 13 07:36:18.822096 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.822061 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" Jul 13 07:36:18.822301 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.822092 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" event={"ID":"b13dd2a1-1c28-4544-ae21-653522a2ec41","Type":"ContainerDied","Data":"bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e"} Jul 13 07:36:18.822301 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.822154 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn" event={"ID":"b13dd2a1-1c28-4544-ae21-653522a2ec41","Type":"ContainerDied","Data":"835d4c23bb6d4a399829053d0eccfde87ff1a0f02599a5d58c1b49f84ea2d40f"} Jul 13 07:36:18.822301 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.822181 2583 scope.go:117] "RemoveContainer" containerID="bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e" Jul 13 07:36:18.827534 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.827513 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-certs\") pod \"b13dd2a1-1c28-4544-ae21-653522a2ec41\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " Jul 13 07:36:18.827632 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.827620 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-socket\") pod \"b13dd2a1-1c28-4544-ae21-653522a2ec41\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " Jul 13 07:36:18.827681 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.827666 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-envoy\") pod \"b13dd2a1-1c28-4544-ae21-653522a2ec41\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " Jul 13 07:36:18.827720 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.827703 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b13dd2a1-1c28-4544-ae21-653522a2ec41-istiod-ca-cert\") pod \"b13dd2a1-1c28-4544-ae21-653522a2ec41\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " Jul 13 07:36:18.827768 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.827744 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-token\") pod \"b13dd2a1-1c28-4544-ae21-653522a2ec41\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " Jul 13 07:36:18.827768 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.827748 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b13dd2a1-1c28-4544-ae21-653522a2ec41" (UID: "b13dd2a1-1c28-4544-ae21-653522a2ec41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:36:18.827869 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.827774 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-podinfo\") pod \"b13dd2a1-1c28-4544-ae21-653522a2ec41\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " Jul 13 07:36:18.827917 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.827865 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b13dd2a1-1c28-4544-ae21-653522a2ec41" (UID: "b13dd2a1-1c28-4544-ae21-653522a2ec41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:36:18.827970 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.827929 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vw68\" (UniqueName: \"kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-kube-api-access-8vw68\") pod \"b13dd2a1-1c28-4544-ae21-653522a2ec41\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " Jul 13 07:36:18.828027 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.827993 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-data\") pod \"b13dd2a1-1c28-4544-ae21-653522a2ec41\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " Jul 13 07:36:18.828087 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.828031 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-credential-socket\") pod \"b13dd2a1-1c28-4544-ae21-653522a2ec41\" (UID: \"b13dd2a1-1c28-4544-ae21-653522a2ec41\") " Jul 13 07:36:18.828087 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.828074 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b13dd2a1-1c28-4544-ae21-653522a2ec41-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b13dd2a1-1c28-4544-ae21-653522a2ec41" (UID: "b13dd2a1-1c28-4544-ae21-653522a2ec41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:36:18.828449 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.828429 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:18.828622 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.828591 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b13dd2a1-1c28-4544-ae21-653522a2ec41-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:18.828622 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.828617 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:18.828819 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.828424 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b13dd2a1-1c28-4544-ae21-653522a2ec41" (UID: "b13dd2a1-1c28-4544-ae21-653522a2ec41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:36:18.828987 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.828960 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-data" (OuterVolumeSpecName: "istio-data") pod "b13dd2a1-1c28-4544-ae21-653522a2ec41" (UID: "b13dd2a1-1c28-4544-ae21-653522a2ec41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:36:18.830068 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.830036 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-token" (OuterVolumeSpecName: "istio-token") pod "b13dd2a1-1c28-4544-ae21-653522a2ec41" (UID: "b13dd2a1-1c28-4544-ae21-653522a2ec41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:36:18.830160 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.830077 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b13dd2a1-1c28-4544-ae21-653522a2ec41" (UID: "b13dd2a1-1c28-4544-ae21-653522a2ec41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:36:18.830160 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.830128 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-kube-api-access-8vw68" (OuterVolumeSpecName: "kube-api-access-8vw68") pod "b13dd2a1-1c28-4544-ae21-653522a2ec41" (UID: "b13dd2a1-1c28-4544-ae21-653522a2ec41"). InnerVolumeSpecName "kube-api-access-8vw68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:36:18.830318 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.830170 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b13dd2a1-1c28-4544-ae21-653522a2ec41" (UID: "b13dd2a1-1c28-4544-ae21-653522a2ec41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:36:18.831937 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.831923 2583 scope.go:117] "RemoveContainer" containerID="2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521" Jul 13 07:36:18.849672 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.849655 2583 scope.go:117] "RemoveContainer" containerID="bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e" Jul 13 07:36:18.849953 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:36:18.849930 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e\": container with ID starting with bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e not found: ID does not exist" containerID="bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e" Jul 13 07:36:18.850034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.849963 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e"} err="failed to get container status \"bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e\": rpc error: code = NotFound desc = could not find container \"bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e\": container with ID starting with bb75fa48175c36781cbe6ae0235d937593e17dec453960357fb9a3507d3e025e not found: ID does not exist" Jul 13 07:36:18.850034 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.849981 2583 scope.go:117] "RemoveContainer" containerID="2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521" Jul 13 07:36:18.850259 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:36:18.850199 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521\": container with ID starting with 2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521 not found: ID does not exist" containerID="2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521" Jul 13 07:36:18.850259 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.850248 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521"} err="failed to get container status \"2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521\": rpc error: code = NotFound desc = could not find container \"2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521\": container with ID starting with 2c14ee9ffccc86273510ec869f3daf1444ef20f20d6117a0bbf109265e77e521 not found: ID does not exist" Jul 13 07:36:18.930094 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.930068 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vw68\" (UniqueName: \"kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-kube-api-access-8vw68\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:18.930524 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.930097 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:18.930524 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.930111 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:18.930524 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.930123 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:18.930524 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.930136 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:18.930524 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:18.930148 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b13dd2a1-1c28-4544-ae21-653522a2ec41-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:36:19.145250 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:19.145219 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn"] Jul 13 07:36:19.149047 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:19.149023 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-780e06-openshift-default-55d6749fcc-49rnn"] Jul 13 07:36:19.429355 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:19.429283 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" path="/var/lib/kubelet/pods/b13dd2a1-1c28-4544-ae21-653522a2ec41/volumes" Jul 13 07:36:22.239090 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.239062 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:22.565172 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.565090 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:22.836616 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.836544 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7"] Jul 13 07:36:22.837249 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.837230 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" Jul 13 07:36:22.837302 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.837253 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" Jul 13 07:36:22.837302 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.837274 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" Jul 13 07:36:22.837302 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.837282 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" Jul 13 07:36:22.837413 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.837401 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" Jul 13 07:36:22.837448 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.837420 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b13dd2a1-1c28-4544-ae21-653522a2ec41" containerName="istio-proxy" Jul 13 07:36:22.842026 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.842006 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:22.844660 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.844638 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3ee270-openshift-default-dockercfg-7vkkq\"" Jul 13 07:36:22.853021 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.852993 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7"] Jul 13 07:36:22.874864 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.874823 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:22.965989 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.965913 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx9xd\" (UniqueName: \"kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-kube-api-access-qx9xd\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:22.965989 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.965964 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-podinfo\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:22.966405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.966006 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istiod-ca-cert\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:22.966405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.966082 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-data\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:22.966405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.966146 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-socket\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:22.966405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.966278 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-envoy\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:22.966405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.966357 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-token\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:22.966405 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.966390 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-certs\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:22.966721 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:22.966425 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-credential-socket\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.067682 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.067653 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-socket\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.067802 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.067701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-envoy\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.067802 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.067736 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-token\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.067802 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.067756 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-certs\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.067802 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.067784 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-credential-socket\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.067993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.067815 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx9xd\" (UniqueName: \"kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-kube-api-access-qx9xd\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.067993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.067841 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-podinfo\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.067993 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.067869 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istiod-ca-cert\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.068151 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.068121 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-socket\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.068238 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.068168 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-data\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.068238 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.068191 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-certs\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.068349 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.068312 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-credential-socket\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.068530 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.068506 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-data\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.068653 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.068633 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istiod-ca-cert\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.070096 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.070078 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-envoy\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.070460 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.070436 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-podinfo\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.078459 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.078431 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-token\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.078897 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.078874 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx9xd\" (UniqueName: \"kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-kube-api-access-qx9xd\") pod \"e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.109273 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.109193 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:23.153095 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.153072 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:23.288506 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.288478 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7"] Jul 13 07:36:23.290251 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:36:23.290217 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df8e1c3_d28e_4396_b3f5_fa7eb2cfafe1.slice/crio-ff124e11cdc98ff614e229e44d762002856f975da433457e6b1f9e84bd76e343 WatchSource:0}: Error finding container ff124e11cdc98ff614e229e44d762002856f975da433457e6b1f9e84bd76e343: Status 404 returned error can't find the container with id ff124e11cdc98ff614e229e44d762002856f975da433457e6b1f9e84bd76e343 Jul 13 07:36:23.292449 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.292414 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:23.292548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.292499 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:23.292548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.292543 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:23.849610 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.849568 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" event={"ID":"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1","Type":"ContainerStarted","Data":"963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708"} Jul 13 07:36:23.849783 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.849641 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" event={"ID":"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1","Type":"ContainerStarted","Data":"ff124e11cdc98ff614e229e44d762002856f975da433457e6b1f9e84bd76e343"} Jul 13 07:36:23.874280 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:23.874229 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" podStartSLOduration=1.874195414 podStartE2EDuration="1.874195414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:36:23.870718596 +0000 UTC m=+2037.045222408" watchObservedRunningTime="2026-07-13 07:36:23.874195414 +0000 UTC m=+2037.048699220" Jul 13 07:36:24.154250 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:24.154155 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:24.158644 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:24.158616 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:24.855335 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:24.855300 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:24.855961 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:24.855944 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:25.923473 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:25.923444 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:25.999580 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:25.999548 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28"] Jul 13 07:36:26.004699 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.004679 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.012219 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.012183 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-3ee270-serving-cert\"" Jul 13 07:36:26.020380 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.020047 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28"] Jul 13 07:36:26.042895 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.042868 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:26.114370 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.114341 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr2g8\" (UniqueName: \"kubernetes.io/projected/e021be7f-63b4-4f3c-a820-b610c35c37df-kube-api-access-pr2g8\") pod \"maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.114555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.114442 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e021be7f-63b4-4f3c-a820-b610c35c37df-maas-api-tls\") pod \"maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.114555 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.114476 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e021be7f-63b4-4f3c-a820-b610c35c37df-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.134344 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.134303 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:26.215503 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.215481 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr2g8\" (UniqueName: \"kubernetes.io/projected/e021be7f-63b4-4f3c-a820-b610c35c37df-kube-api-access-pr2g8\") pod \"maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.215672 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.215643 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e021be7f-63b4-4f3c-a820-b610c35c37df-maas-api-tls\") pod \"maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.215731 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.215716 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e021be7f-63b4-4f3c-a820-b610c35c37df-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.216374 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.216355 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e021be7f-63b4-4f3c-a820-b610c35c37df-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.218300 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.218282 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e021be7f-63b4-4f3c-a820-b610c35c37df-maas-api-tls\") pod \"maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.230814 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.230794 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr2g8\" (UniqueName: \"kubernetes.io/projected/e021be7f-63b4-4f3c-a820-b610c35c37df-kube-api-access-pr2g8\") pod \"maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.232442 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.232418 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:26.322849 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.322815 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:26.514065 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.508854 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28"] Jul 13 07:36:26.606922 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.606891 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:26.704245 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.704097 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:26.868153 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:26.868072 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" event={"ID":"e021be7f-63b4-4f3c-a820-b610c35c37df","Type":"ContainerStarted","Data":"48dbfbf4f59e862e0263780e608e21e66b7ca181b47decf713d573196c6ab151"} Jul 13 07:36:27.876127 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:27.876086 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" event={"ID":"e021be7f-63b4-4f3c-a820-b610c35c37df","Type":"ContainerStarted","Data":"6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e"} Jul 13 07:36:27.876487 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:27.876181 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:30.278890 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:30.278861 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:30.892560 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:30.892527 2583 generic.go:356] "Generic (PLEG): container finished" podID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerID="c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084" exitCode=0 Jul 13 07:36:30.892707 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:30.892598 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" event={"ID":"a674627c-1fc4-48ac-bb29-e9d07c84a3d3","Type":"ContainerDied","Data":"c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084"} Jul 13 07:36:30.892941 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:30.892926 2583 scope.go:117] "RemoveContainer" containerID="c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084" Jul 13 07:36:30.893938 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:30.893910 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:30.893998 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:30.893980 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:30.894041 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:30.894019 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:30.920069 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:30.920026 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" podStartSLOduration=5.337330158 podStartE2EDuration="5.920009421s" totalImagesPullingTime="582.679263ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:36:27.906393859 +0000 UTC m=+2041.080897690" watchObservedRunningTime="2026-07-13 07:36:30.920009421 +0000 UTC m=+2044.094513229" Jul 13 07:36:31.809458 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:31.809425 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:31.904467 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:31.904422 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" event={"ID":"a674627c-1fc4-48ac-bb29-e9d07c84a3d3","Type":"ContainerStarted","Data":"7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b"} Jul 13 07:36:32.684728 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:32.684700 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:32.685089 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:32.685061 2583 patch_prober.go:28] interesting pod/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 07:36:32.685306 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:32.685278 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 07:36:33.683975 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:33.683933 2583 patch_prober.go:28] interesting pod/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 07:36:33.684504 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:33.683998 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 07:36:33.896504 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:33.896468 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:36:34.684244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:34.684183 2583 patch_prober.go:28] interesting pod/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 07:36:34.684748 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:34.684268 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 07:36:35.689800 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:35.689768 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:35.691887 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:35.691868 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:35.692844 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:35.692830 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:36:39.428052 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:39.428020 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:39.527144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:39.527115 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:39.856513 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:39.856442 2583 patch_prober.go:28] interesting pod/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 13 07:36:39.856674 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:39.856506 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 13 07:36:39.960952 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:39.960927 2583 generic.go:356] "Generic (PLEG): container finished" podID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerID="963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708" exitCode=0 Jul 13 07:36:39.961080 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:39.961016 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" event={"ID":"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1","Type":"ContainerDied","Data":"963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708"} Jul 13 07:36:39.961425 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:39.961411 2583 scope.go:117] "RemoveContainer" containerID="963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708" Jul 13 07:36:39.962402 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:39.962372 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:39.962494 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:39.962438 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:39.962494 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:39.962473 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 07:36:40.976101 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:40.976063 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" event={"ID":"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1","Type":"ContainerStarted","Data":"e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273"} Jul 13 07:36:41.153983 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:41.153949 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:41.154170 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:41.154146 2583 patch_prober.go:28] interesting pod/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 13 07:36:41.154239 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:41.154215 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 13 07:36:42.153886 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:42.153838 2583 patch_prober.go:28] interesting pod/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 13 07:36:42.154287 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:42.153905 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 13 07:36:43.154294 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:43.154262 2583 patch_prober.go:28] interesting pod/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 13 07:36:43.154655 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:43.154316 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 13 07:36:44.153725 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:44.153691 2583 patch_prober.go:28] interesting pod/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 13 07:36:44.153933 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:44.153745 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 13 07:36:45.157505 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:45.157475 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:45.157920 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:45.157844 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:45.158459 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:45.158440 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:36:47.361304 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:47.361273 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:47.463885 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:47.463851 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:48.613386 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:48.613355 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:36:48.680283 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:36:48.680255 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:38:43.518833 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:38:43.518799 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:35.175695 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:35.175658 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:36.839407 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:36.839378 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:41.666045 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:41.666013 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf"] Jul 13 07:39:41.666629 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:41.666247 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerName="istio-proxy" containerID="cri-o://7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b" gracePeriod=30 Jul 13 07:39:41.759584 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:41.759552 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:41.821118 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:41.821084 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:42.121103 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:42.121071 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:42.540925 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:42.540890 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:42.651880 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:42.651848 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:42.748824 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:42.748793 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:42.868172 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:42.868095 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28"] Jul 13 07:39:42.868334 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:42.868312 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" podUID="e021be7f-63b4-4f3c-a820-b610c35c37df" containerName="maas-api" containerID="cri-o://6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e" gracePeriod=30 Jul 13 07:39:42.891568 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:42.891542 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:43.121059 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.121010 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:39:43.200393 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.200366 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr2g8\" (UniqueName: \"kubernetes.io/projected/e021be7f-63b4-4f3c-a820-b610c35c37df-kube-api-access-pr2g8\") pod \"e021be7f-63b4-4f3c-a820-b610c35c37df\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " Jul 13 07:39:43.200524 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.200432 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e021be7f-63b4-4f3c-a820-b610c35c37df-openshift-service-ca\") pod \"e021be7f-63b4-4f3c-a820-b610c35c37df\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " Jul 13 07:39:43.200614 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.200590 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e021be7f-63b4-4f3c-a820-b610c35c37df-maas-api-tls\") pod \"e021be7f-63b4-4f3c-a820-b610c35c37df\" (UID: \"e021be7f-63b4-4f3c-a820-b610c35c37df\") " Jul 13 07:39:43.200740 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.200720 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e021be7f-63b4-4f3c-a820-b610c35c37df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e021be7f-63b4-4f3c-a820-b610c35c37df" (UID: "e021be7f-63b4-4f3c-a820-b610c35c37df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:39:43.200938 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.200915 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e021be7f-63b4-4f3c-a820-b610c35c37df-openshift-service-ca\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:43.202412 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.202391 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e021be7f-63b4-4f3c-a820-b610c35c37df-kube-api-access-pr2g8" (OuterVolumeSpecName: "kube-api-access-pr2g8") pod "e021be7f-63b4-4f3c-a820-b610c35c37df" (UID: "e021be7f-63b4-4f3c-a820-b610c35c37df"). InnerVolumeSpecName "kube-api-access-pr2g8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:39:43.202486 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.202414 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e021be7f-63b4-4f3c-a820-b610c35c37df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e021be7f-63b4-4f3c-a820-b610c35c37df" (UID: "e021be7f-63b4-4f3c-a820-b610c35c37df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:39:43.301961 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.301939 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr2g8\" (UniqueName: \"kubernetes.io/projected/e021be7f-63b4-4f3c-a820-b610c35c37df-kube-api-access-pr2g8\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:43.301961 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.301959 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e021be7f-63b4-4f3c-a820-b610c35c37df-maas-api-tls\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:43.911155 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.911124 2583 generic.go:356] "Generic (PLEG): container finished" podID="e021be7f-63b4-4f3c-a820-b610c35c37df" containerID="6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e" exitCode=0 Jul 13 07:39:43.911548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.911161 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" event={"ID":"e021be7f-63b4-4f3c-a820-b610c35c37df","Type":"ContainerDied","Data":"6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e"} Jul 13 07:39:43.911548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.911176 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" Jul 13 07:39:43.911548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.911197 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28" event={"ID":"e021be7f-63b4-4f3c-a820-b610c35c37df","Type":"ContainerDied","Data":"48dbfbf4f59e862e0263780e608e21e66b7ca181b47decf713d573196c6ab151"} Jul 13 07:39:43.911548 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.911246 2583 scope.go:117] "RemoveContainer" containerID="6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e" Jul 13 07:39:43.919621 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.919602 2583 scope.go:117] "RemoveContainer" containerID="6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e" Jul 13 07:39:43.919861 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:39:43.919843 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e\": container with ID starting with 6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e not found: ID does not exist" containerID="6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e" Jul 13 07:39:43.919907 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.919874 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e"} err="failed to get container status \"6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e\": rpc error: code = NotFound desc = could not find container \"6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e\": container with ID starting with 6d9cbfea5dc46930a4b9e8ad926d4e97c519253daaecffbd32230a6fda790c6e not found: ID does not exist" Jul 13 07:39:43.930581 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.930556 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28"] Jul 13 07:39:43.937992 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:43.937970 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee270-848d6476f6-n7c28"] Jul 13 07:39:45.428439 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:45.428407 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e021be7f-63b4-4f3c-a820-b610c35c37df" path="/var/lib/kubelet/pods/e021be7f-63b4-4f3c-a820-b610c35c37df/volumes" Jul 13 07:39:46.920067 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.920045 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:39:46.928809 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.928781 2583 generic.go:356] "Generic (PLEG): container finished" podID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerID="7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b" exitCode=0 Jul 13 07:39:46.928913 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.928826 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" Jul 13 07:39:46.928913 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.928832 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" event={"ID":"a674627c-1fc4-48ac-bb29-e9d07c84a3d3","Type":"ContainerDied","Data":"7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b"} Jul 13 07:39:46.928913 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.928893 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf" event={"ID":"a674627c-1fc4-48ac-bb29-e9d07c84a3d3","Type":"ContainerDied","Data":"6d19ca57301d9d5f884cb477409414bcd7e6e13fddf698a2292a01c6629376dc"} Jul 13 07:39:46.929050 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.928923 2583 scope.go:117] "RemoveContainer" containerID="7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b" Jul 13 07:39:46.937444 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.937425 2583 scope.go:117] "RemoveContainer" containerID="c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084" Jul 13 07:39:46.946944 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.946921 2583 scope.go:117] "RemoveContainer" containerID="7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b" Jul 13 07:39:46.947247 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:39:46.947227 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b\": container with ID starting with 7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b not found: ID does not exist" containerID="7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b" Jul 13 07:39:46.947333 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.947256 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b"} err="failed to get container status \"7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b\": rpc error: code = NotFound desc = could not find container \"7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b\": container with ID starting with 7bbda2094e0dd9e023826f295844e903027c57f5f63db14ab2daa7d536ad054b not found: ID does not exist" Jul 13 07:39:46.947333 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.947279 2583 scope.go:117] "RemoveContainer" containerID="c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084" Jul 13 07:39:46.947504 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:39:46.947488 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084\": container with ID starting with c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084 not found: ID does not exist" containerID="c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084" Jul 13 07:39:46.947562 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:46.947510 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084"} err="failed to get container status \"c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084\": rpc error: code = NotFound desc = could not find container \"c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084\": container with ID starting with c014923bc39dc8472695209655241c0afddf8bc11b64015be82c1ebb6d413084 not found: ID does not exist" Jul 13 07:39:47.029473 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029451 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-envoy\") pod \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " Jul 13 07:39:47.029552 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029478 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-data\") pod \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " Jul 13 07:39:47.029552 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029493 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istiod-ca-cert\") pod \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " Jul 13 07:39:47.029552 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029515 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-podinfo\") pod \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " Jul 13 07:39:47.029679 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029563 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh6br\" (UniqueName: \"kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-kube-api-access-qh6br\") pod \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " Jul 13 07:39:47.029679 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029596 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-credential-socket\") pod \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " Jul 13 07:39:47.029679 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029637 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-socket\") pod \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " Jul 13 07:39:47.029844 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029790 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-certs\") pod \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " Jul 13 07:39:47.029844 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029840 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-token\") pod \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\" (UID: \"a674627c-1fc4-48ac-bb29-e9d07c84a3d3\") " Jul 13 07:39:47.029954 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029882 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a674627c-1fc4-48ac-bb29-e9d07c84a3d3" (UID: "a674627c-1fc4-48ac-bb29-e9d07c84a3d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:39:47.030011 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.029989 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a674627c-1fc4-48ac-bb29-e9d07c84a3d3" (UID: "a674627c-1fc4-48ac-bb29-e9d07c84a3d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:39:47.030191 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.030152 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a674627c-1fc4-48ac-bb29-e9d07c84a3d3" (UID: "a674627c-1fc4-48ac-bb29-e9d07c84a3d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:39:47.030191 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.030171 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a674627c-1fc4-48ac-bb29-e9d07c84a3d3" (UID: "a674627c-1fc4-48ac-bb29-e9d07c84a3d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:39:47.030378 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.030235 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:47.030378 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.030255 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:47.030458 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.030400 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "a674627c-1fc4-48ac-bb29-e9d07c84a3d3" (UID: "a674627c-1fc4-48ac-bb29-e9d07c84a3d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:39:47.031660 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.031639 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a674627c-1fc4-48ac-bb29-e9d07c84a3d3" (UID: "a674627c-1fc4-48ac-bb29-e9d07c84a3d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:39:47.031933 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.031908 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "a674627c-1fc4-48ac-bb29-e9d07c84a3d3" (UID: "a674627c-1fc4-48ac-bb29-e9d07c84a3d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:39:47.032025 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.031995 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-kube-api-access-qh6br" (OuterVolumeSpecName: "kube-api-access-qh6br") pod "a674627c-1fc4-48ac-bb29-e9d07c84a3d3" (UID: "a674627c-1fc4-48ac-bb29-e9d07c84a3d3"). InnerVolumeSpecName "kube-api-access-qh6br". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:39:47.032025 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.031997 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a674627c-1fc4-48ac-bb29-e9d07c84a3d3" (UID: "a674627c-1fc4-48ac-bb29-e9d07c84a3d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:39:47.131394 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.131364 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:47.131394 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.131388 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:47.131394 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.131396 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:47.131394 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.131403 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:47.131650 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.131413 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:47.131650 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.131421 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh6br\" (UniqueName: \"kubernetes.io/projected/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-kube-api-access-qh6br\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:47.131650 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.131429 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a674627c-1fc4-48ac-bb29-e9d07c84a3d3-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:47.250070 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.250039 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf"] Jul 13 07:39:47.255569 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.255545 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-cc53e7-openshift-default-5d94c96779-6brzf"] Jul 13 07:39:47.428636 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:47.428608 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" path="/var/lib/kubelet/pods/a674627c-1fc4-48ac-bb29-e9d07c84a3d3/volumes" Jul 13 07:39:48.853625 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:48.853597 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:48.954109 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:48.954077 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:49.121387 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:49.121284 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7"] Jul 13 07:39:49.121540 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:49.121485 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" containerID="cri-o://e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273" gracePeriod=30 Jul 13 07:39:49.138351 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:49.138324 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:39:54.378463 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.378433 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:39:54.500589 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.500555 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-certs\") pod \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " Jul 13 07:39:54.500589 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.500586 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-podinfo\") pod \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " Jul 13 07:39:54.500788 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.500615 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-data\") pod \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " Jul 13 07:39:54.500788 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.500651 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx9xd\" (UniqueName: \"kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-kube-api-access-qx9xd\") pod \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " Jul 13 07:39:54.500788 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.500670 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istiod-ca-cert\") pod \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " Jul 13 07:39:54.500788 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.500692 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-token\") pod \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " Jul 13 07:39:54.500999 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.500788 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" (UID: "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:39:54.500999 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.500782 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-envoy\") pod \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " Jul 13 07:39:54.500999 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.500848 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-credential-socket\") pod \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " Jul 13 07:39:54.500999 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.500883 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-socket\") pod \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\" (UID: \"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1\") " Jul 13 07:39:54.501198 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.501105 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" (UID: "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:39:54.501361 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.501339 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-certs\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:54.501515 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.501499 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-credential-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:54.501599 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.501535 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" (UID: "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:39:54.501669 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.501599 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" (UID: "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:39:54.501791 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.501763 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-data" (OuterVolumeSpecName: "istio-data") pod "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" (UID: "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:39:54.502899 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.502873 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-token" (OuterVolumeSpecName: "istio-token") pod "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" (UID: "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:39:54.503013 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.502961 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" (UID: "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 07:39:54.503373 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.503347 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-kube-api-access-qx9xd" (OuterVolumeSpecName: "kube-api-access-qx9xd") pod "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" (UID: "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1"). InnerVolumeSpecName "kube-api-access-qx9xd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:39:54.503373 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.503347 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" (UID: "9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:39:54.506449 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.506431 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4q48r_87f77770-a3ed-47b1-9cff-27f628d8b85a/manager/0.log" Jul 13 07:39:54.602328 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.602295 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-envoy\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:54.602328 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.602326 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-workload-socket\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:54.602328 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.602335 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-podinfo\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:54.602498 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.602343 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-data\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:54.602498 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.602351 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx9xd\" (UniqueName: \"kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-kube-api-access-qx9xd\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:54.602498 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.602359 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istiod-ca-cert\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:54.602498 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.602367 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1-istio-token\") on node \"ip-10-0-133-57.ec2.internal\" DevicePath \"\"" Jul 13 07:39:54.755070 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.755004 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b7565664c-cprnk_720b787b-8a37-4ce5-beb5-433d5c81961f/manager/0.log" Jul 13 07:39:54.971635 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.971607 2583 generic.go:356] "Generic (PLEG): container finished" podID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerID="e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273" exitCode=0 Jul 13 07:39:54.971750 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.971661 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" Jul 13 07:39:54.971750 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.971686 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" event={"ID":"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1","Type":"ContainerDied","Data":"e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273"} Jul 13 07:39:54.971750 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.971738 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7" event={"ID":"9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1","Type":"ContainerDied","Data":"ff124e11cdc98ff614e229e44d762002856f975da433457e6b1f9e84bd76e343"} Jul 13 07:39:54.971854 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.971761 2583 scope.go:117] "RemoveContainer" containerID="e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273" Jul 13 07:39:54.981138 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.981124 2583 scope.go:117] "RemoveContainer" containerID="963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708" Jul 13 07:39:54.988980 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.988964 2583 scope.go:117] "RemoveContainer" containerID="e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273" Jul 13 07:39:54.989256 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:39:54.989186 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273\": container with ID starting with e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273 not found: ID does not exist" containerID="e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273" Jul 13 07:39:54.989256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.989227 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273"} err="failed to get container status \"e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273\": rpc error: code = NotFound desc = could not find container \"e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273\": container with ID starting with e83af71960fb16270463990b76fa357e837b599def293524b4f8271cde72d273 not found: ID does not exist" Jul 13 07:39:54.989256 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.989243 2583 scope.go:117] "RemoveContainer" containerID="963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708" Jul 13 07:39:54.989459 ip-10-0-133-57 kubenswrapper[2583]: E0713 07:39:54.989441 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708\": container with ID starting with 963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708 not found: ID does not exist" containerID="963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708" Jul 13 07:39:54.989496 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.989466 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708"} err="failed to get container status \"963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708\": rpc error: code = NotFound desc = could not find container \"963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708\": container with ID starting with 963eee6ab230062c53826d3b062f2a1580faea7f6c460cb8872a0b2a57a33708 not found: ID does not exist" Jul 13 07:39:55.000284 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:54.997460 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7"] Jul 13 07:39:55.006360 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:55.006256 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3ee270-openshift-default-7cd87fddf5-5v6r7"] Jul 13 07:39:55.380948 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:55.380876 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7487689bdd-s84l5_1bc19e2c-748b-40e2-afef-60e7f005254c/copy-manifests/0.log" Jul 13 07:39:55.388130 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:55.388106 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7487689bdd-s84l5_1bc19e2c-748b-40e2-afef-60e7f005254c/manager/0.log" Jul 13 07:39:55.428772 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:55.428746 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" path="/var/lib/kubelet/pods/9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1/volumes" Jul 13 07:39:56.909341 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:56.909302 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-x8vdw_d0a15549-9350-4fe4-a515-77f42e19c9d0/manager/0.log" Jul 13 07:39:57.016776 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:57.016751 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-ct8bf_4164614c-0b92-48e9-a3a3-432a3711ddee/kuadrant-console-plugin/0.log" Jul 13 07:39:57.851602 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:57.851575 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vnf99_105d4deb-f1d8-42cc-8851-d3987c1d440b/istio-proxy/0.log" Jul 13 07:39:58.317544 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:58.317519 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ppflf_c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2/istio-proxy/1.log" Jul 13 07:39:58.442180 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:58.442151 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-bb8j4_540a0df8-8613-41c9-8218-e0c1fbaa2e4a/payload-pre-processing/0.log" Jul 13 07:39:58.563841 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:58.563809 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-thlj2_68e30d4a-f566-4cfe-a2c5-4f61be32aead/payload-processing/0.log" Jul 13 07:39:59.020944 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:59.020919 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-dfwqb_1645ff78-f8ee-4259-8921-708b5c7fd45b/main/0.log" Jul 13 07:39:59.235559 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:59.235536 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-7xtc9_52398baf-ed13-4e45-9968-5067bb1046f6/main/0.log" Jul 13 07:39:59.346432 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:39:59.346358 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfh4j_4f756c19-d5df-49e5-95f0-f0b7104b5381/main/0.log" Jul 13 07:40:04.112510 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:04.112474 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:40:04.210159 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:04.210124 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" limit=1024 Jul 13 07:40:06.962616 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:06.962585 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b2qc9_b6d741e6-fdd4-442e-9940-2588e3ef9419/global-pull-secret-syncer/0.log" Jul 13 07:40:07.098902 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:07.098876 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-msp7f_b1295416-be49-4b38-9602-0c28e9e90055/konnectivity-agent/0.log" Jul 13 07:40:07.193448 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:07.193423 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-57.ec2.internal_b573b3797697c17ef8fefe9494186a99/haproxy/0.log" Jul 13 07:40:12.973725 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:12.973695 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-x8vdw_d0a15549-9350-4fe4-a515-77f42e19c9d0/manager/0.log" Jul 13 07:40:13.004756 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:13.004728 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-ct8bf_4164614c-0b92-48e9-a3a3-432a3711ddee/kuadrant-console-plugin/0.log" Jul 13 07:40:14.863403 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:14.863373 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f020a5ad-4a70-418a-a7bd-667af16bf35d/alertmanager/0.log" Jul 13 07:40:14.914959 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:14.914931 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f020a5ad-4a70-418a-a7bd-667af16bf35d/config-reloader/0.log" Jul 13 07:40:14.980808 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:14.980784 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f020a5ad-4a70-418a-a7bd-667af16bf35d/kube-rbac-proxy-web/0.log" Jul 13 07:40:15.030487 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.030458 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f020a5ad-4a70-418a-a7bd-667af16bf35d/kube-rbac-proxy/0.log" Jul 13 07:40:15.085601 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.085539 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f020a5ad-4a70-418a-a7bd-667af16bf35d/kube-rbac-proxy-metric/0.log" Jul 13 07:40:15.137947 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.137909 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f020a5ad-4a70-418a-a7bd-667af16bf35d/prom-label-proxy/0.log" Jul 13 07:40:15.173019 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.172995 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f020a5ad-4a70-418a-a7bd-667af16bf35d/init-config-reloader/0.log" Jul 13 07:40:15.268383 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.268357 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-6cvmw_618475e3-496a-4bdc-bea8-8222dc14a179/kube-state-metrics/0.log" Jul 13 07:40:15.308762 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.308742 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-6cvmw_618475e3-496a-4bdc-bea8-8222dc14a179/kube-rbac-proxy-main/0.log" Jul 13 07:40:15.344318 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.344259 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-6cvmw_618475e3-496a-4bdc-bea8-8222dc14a179/kube-rbac-proxy-self/0.log" Jul 13 07:40:15.382144 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.382124 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5d66dd77b8-7svz5_8694a18f-2def-4299-b570-b0c4df0c21cc/metrics-server/0.log" Jul 13 07:40:15.428588 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.428566 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-mmsrp_800b08e1-c950-4dcf-a73e-3eebdfc3f5d1/monitoring-plugin/0.log" Jul 13 07:40:15.710427 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.710399 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8lltp_6b2a8f04-ea30-4461-a460-7fb3a891e063/node-exporter/0.log" Jul 13 07:40:15.737980 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.737958 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8lltp_6b2a8f04-ea30-4461-a460-7fb3a891e063/kube-rbac-proxy/0.log" Jul 13 07:40:15.762365 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.762335 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8lltp_6b2a8f04-ea30-4461-a460-7fb3a891e063/init-textfile/0.log" Jul 13 07:40:15.789413 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.789394 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-t8lb4_f47ad689-c9b0-4f6f-9871-adec98f855fe/kube-rbac-proxy-main/0.log" Jul 13 07:40:15.834267 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.834248 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-t8lb4_f47ad689-c9b0-4f6f-9871-adec98f855fe/kube-rbac-proxy-self/0.log" Jul 13 07:40:15.859342 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.859291 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-t8lb4_f47ad689-c9b0-4f6f-9871-adec98f855fe/openshift-state-metrics/0.log" Jul 13 07:40:15.899258 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.899244 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bbd6c48-eb6d-4ef6-ac41-981999cd95fe/prometheus/0.log" Jul 13 07:40:15.929899 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.929879 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bbd6c48-eb6d-4ef6-ac41-981999cd95fe/config-reloader/0.log" Jul 13 07:40:15.956762 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.956744 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bbd6c48-eb6d-4ef6-ac41-981999cd95fe/thanos-sidecar/0.log" Jul 13 07:40:15.987693 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:15.987677 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bbd6c48-eb6d-4ef6-ac41-981999cd95fe/kube-rbac-proxy-web/0.log" Jul 13 07:40:16.019441 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.019422 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bbd6c48-eb6d-4ef6-ac41-981999cd95fe/kube-rbac-proxy/0.log" Jul 13 07:40:16.044813 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.044796 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bbd6c48-eb6d-4ef6-ac41-981999cd95fe/kube-rbac-proxy-thanos/0.log" Jul 13 07:40:16.071392 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.071369 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bbd6c48-eb6d-4ef6-ac41-981999cd95fe/init-config-reloader/0.log" Jul 13 07:40:16.117169 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.117110 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-zrg6c_9cb1c18f-81e6-4761-85cc-c65dc0774aaa/prometheus-operator/0.log" Jul 13 07:40:16.137882 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.137818 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-zrg6c_9cb1c18f-81e6-4761-85cc-c65dc0774aaa/kube-rbac-proxy/0.log" Jul 13 07:40:16.200554 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.200538 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5fd557576d-xnvk5_2fc1a8f5-38b0-4326-8722-451f9258c1f0/telemeter-client/0.log" Jul 13 07:40:16.224078 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.224052 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5fd557576d-xnvk5_2fc1a8f5-38b0-4326-8722-451f9258c1f0/reload/0.log" Jul 13 07:40:16.255661 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.255639 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5fd557576d-xnvk5_2fc1a8f5-38b0-4326-8722-451f9258c1f0/kube-rbac-proxy/0.log" Jul 13 07:40:16.297214 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.297182 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d87b465-6tjct_84ddf01b-41e7-4c8d-8cc5-105691b08bd7/thanos-query/0.log" Jul 13 07:40:16.323742 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.323727 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d87b465-6tjct_84ddf01b-41e7-4c8d-8cc5-105691b08bd7/kube-rbac-proxy-web/0.log" Jul 13 07:40:16.351686 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.351672 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d87b465-6tjct_84ddf01b-41e7-4c8d-8cc5-105691b08bd7/kube-rbac-proxy/0.log" Jul 13 07:40:16.377977 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.377927 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d87b465-6tjct_84ddf01b-41e7-4c8d-8cc5-105691b08bd7/prom-label-proxy/0.log" Jul 13 07:40:16.402654 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.402630 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d87b465-6tjct_84ddf01b-41e7-4c8d-8cc5-105691b08bd7/kube-rbac-proxy-rules/0.log" Jul 13 07:40:16.423615 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.423594 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d87b465-6tjct_84ddf01b-41e7-4c8d-8cc5-105691b08bd7/kube-rbac-proxy-metrics/0.log" Jul 13 07:40:16.818697 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.818669 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7"] Jul 13 07:40:16.819113 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819101 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerName="istio-proxy" Jul 13 07:40:16.819113 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819114 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerName="istio-proxy" Jul 13 07:40:16.819230 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819122 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerName="istio-proxy" Jul 13 07:40:16.819230 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819128 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerName="istio-proxy" Jul 13 07:40:16.819230 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819150 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e021be7f-63b4-4f3c-a820-b610c35c37df" containerName="maas-api" Jul 13 07:40:16.819230 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819156 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e021be7f-63b4-4f3c-a820-b610c35c37df" containerName="maas-api" Jul 13 07:40:16.819230 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819164 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" Jul 13 07:40:16.819230 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819169 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" Jul 13 07:40:16.819230 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819178 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" Jul 13 07:40:16.819230 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819184 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" Jul 13 07:40:16.819549 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819251 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" Jul 13 07:40:16.819549 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819260 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9df8e1c3-d28e-4396-b3f5-fa7eb2cfafe1" containerName="istio-proxy" Jul 13 07:40:16.819549 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819267 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerName="istio-proxy" Jul 13 07:40:16.819549 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819274 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a674627c-1fc4-48ac-bb29-e9d07c84a3d3" containerName="istio-proxy" Jul 13 07:40:16.819549 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.819281 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e021be7f-63b4-4f3c-a820-b610c35c37df" containerName="maas-api" Jul 13 07:40:16.822861 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.822842 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:16.825437 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.825418 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qmnmv\"/\"openshift-service-ca.crt\"" Jul 13 07:40:16.825519 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.825462 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qmnmv\"/\"kube-root-ca.crt\"" Jul 13 07:40:16.826521 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.826502 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qmnmv\"/\"default-dockercfg-dd7q7\"" Jul 13 07:40:16.831502 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.831484 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7"] Jul 13 07:40:16.989946 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.989919 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-proc\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:16.989946 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.989948 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgnbn\" (UniqueName: \"kubernetes.io/projected/f85694ea-ce89-413f-853d-76ed7f91cb95-kube-api-access-wgnbn\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:16.990318 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.989978 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-sys\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:16.990318 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.990109 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-lib-modules\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:16.990318 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:16.990134 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-podres\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.091450 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.091391 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-lib-modules\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.091450 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.091420 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-podres\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.091450 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.091437 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-proc\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.091622 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.091459 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgnbn\" (UniqueName: \"kubernetes.io/projected/f85694ea-ce89-413f-853d-76ed7f91cb95-kube-api-access-wgnbn\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.091622 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.091504 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-sys\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.091622 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.091528 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-proc\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.091622 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.091554 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-podres\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.091622 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.091559 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-lib-modules\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.091622 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.091583 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f85694ea-ce89-413f-853d-76ed7f91cb95-sys\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.100470 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.100452 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgnbn\" (UniqueName: \"kubernetes.io/projected/f85694ea-ce89-413f-853d-76ed7f91cb95-kube-api-access-wgnbn\") pod \"perf-node-gather-daemonset-qshm7\" (UID: \"f85694ea-ce89-413f-853d-76ed7f91cb95\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.133992 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.133974 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:17.264852 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:17.264828 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7"] Jul 13 07:40:17.266912 ip-10-0-133-57 kubenswrapper[2583]: W0713 07:40:17.266880 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf85694ea_ce89_413f_853d_76ed7f91cb95.slice/crio-e1dc6a471b3a832c75073aa0031f5376b5d7a1f62290ba1a76bd690d9add709c WatchSource:0}: Error finding container e1dc6a471b3a832c75073aa0031f5376b5d7a1f62290ba1a76bd690d9add709c: Status 404 returned error can't find the container with id e1dc6a471b3a832c75073aa0031f5376b5d7a1f62290ba1a76bd690d9add709c Jul 13 07:40:18.085582 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:18.085549 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" event={"ID":"f85694ea-ce89-413f-853d-76ed7f91cb95","Type":"ContainerStarted","Data":"ed1deed5629959852d57b88a5a059eaa04319ae06edcb8aab00e72cae87f4b2c"} Jul 13 07:40:18.086000 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:18.085622 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" event={"ID":"f85694ea-ce89-413f-853d-76ed7f91cb95","Type":"ContainerStarted","Data":"e1dc6a471b3a832c75073aa0031f5376b5d7a1f62290ba1a76bd690d9add709c"} Jul 13 07:40:18.086000 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:18.085683 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:18.105097 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:18.105056 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" podStartSLOduration=2.105042782 podStartE2EDuration="2.105042782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:40:18.10272183 +0000 UTC m=+2271.277225638" watchObservedRunningTime="2026-07-13 07:40:18.105042782 +0000 UTC m=+2271.279546588" Jul 13 07:40:20.396431 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:20.396402 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9ttvg_38de7579-9ff9-4d60-8d06-5534eb04507d/dns/0.log" Jul 13 07:40:20.417825 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:20.417772 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9ttvg_38de7579-9ff9-4d60-8d06-5534eb04507d/kube-rbac-proxy/0.log" Jul 13 07:40:20.635013 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:20.634984 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ztprt_f663e38a-cc1c-4b21-a53a-c40b2ace16c5/dns-node-resolver/0.log" Jul 13 07:40:21.209901 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:21.209870 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sxk2l_ad5f621b-f27f-4abb-a5be-557e0bbceb86/node-ca/0.log" Jul 13 07:40:22.223651 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:22.223604 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vnf99_105d4deb-f1d8-42cc-8851-d3987c1d440b/istio-proxy/0.log" Jul 13 07:40:22.404966 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:22.404943 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ppflf_c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2/istio-proxy/0.log" Jul 13 07:40:22.458667 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:22.458645 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ppflf_c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2/istio-proxy/1.log" Jul 13 07:40:22.509922 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:22.509860 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-bb8j4_540a0df8-8613-41c9-8218-e0c1fbaa2e4a/payload-pre-processing/0.log" Jul 13 07:40:22.580783 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:22.580760 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-thlj2_68e30d4a-f566-4cfe-a2c5-4f61be32aead/payload-processing/0.log" Jul 13 07:40:23.314331 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:23.314305 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4tcb9_10443be9-db7c-47da-8a59-ee567dfa62c8/serve-healthcheck-canary/0.log" Jul 13 07:40:24.046469 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:24.046439 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fpkv5_13b90926-ab1e-4e33-a60e-45d5ba7d5abb/kube-rbac-proxy/0.log" Jul 13 07:40:24.067676 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:24.067653 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fpkv5_13b90926-ab1e-4e33-a60e-45d5ba7d5abb/exporter/0.log" Jul 13 07:40:24.091097 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:24.091070 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fpkv5_13b90926-ab1e-4e33-a60e-45d5ba7d5abb/extractor/0.log" Jul 13 07:40:24.099572 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:24.099556 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-qshm7" Jul 13 07:40:26.706880 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:26.706807 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4q48r_87f77770-a3ed-47b1-9cff-27f628d8b85a/manager/0.log" Jul 13 07:40:26.972411 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:26.972379 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b7565664c-cprnk_720b787b-8a37-4ce5-beb5-433d5c81961f/manager/0.log" Jul 13 07:40:27.242703 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:27.242623 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7487689bdd-s84l5_1bc19e2c-748b-40e2-afef-60e7f005254c/manager/0.log" Jul 13 07:40:27.262388 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:27.262367 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7487689bdd-s84l5_1bc19e2c-748b-40e2-afef-60e7f005254c/copy-manifests/0.log" Jul 13 07:40:27.904685 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:27.904657 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vnf99_105d4deb-f1d8-42cc-8851-d3987c1d440b/istio-proxy/0.log" Jul 13 07:40:28.043871 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:28.043820 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ppflf_c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2/istio-proxy/0.log" Jul 13 07:40:28.100244 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:28.100187 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ppflf_c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2/istio-proxy/1.log" Jul 13 07:40:28.154461 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:28.154432 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-bb8j4_540a0df8-8613-41c9-8218-e0c1fbaa2e4a/payload-pre-processing/0.log" Jul 13 07:40:28.221336 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:28.221315 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-thlj2_68e30d4a-f566-4cfe-a2c5-4f61be32aead/payload-processing/0.log" Jul 13 07:40:28.508420 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:28.508401 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-9f45c7f89-brmql_8191fc53-4ea3-4812-9a74-645682c6b210/manager/0.log" Jul 13 07:40:28.575430 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:28.575407 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2t9jv_23c9757a-37ba-49de-b3e9-4a50a4f84f6d/openshift-lws-operator/0.log" Jul 13 07:40:29.344857 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:29.344817 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vnf99_105d4deb-f1d8-42cc-8851-d3987c1d440b/istio-proxy/0.log" Jul 13 07:40:29.481351 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:29.481326 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ppflf_c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2/istio-proxy/0.log" Jul 13 07:40:29.538441 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:29.538414 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ppflf_c8dd9b2e-20ab-4088-89d2-a674fd3a1aa2/istio-proxy/1.log" Jul 13 07:40:29.586633 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:29.586607 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-bb8j4_540a0df8-8613-41c9-8218-e0c1fbaa2e4a/payload-pre-processing/0.log" Jul 13 07:40:29.648105 ip-10-0-133-57 kubenswrapper[2583]: I0713 07:40:29.648048 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-thlj2_68e30d4a-f566-4cfe-a2c5-4f61be32aead/payload-processing/0.log"