Jul 03 10:05:21.818434 ip-10-0-135-154 systemd[1]: Starting Kubernetes Kubelet... Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.224996 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229336 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229351 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229355 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229358 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229361 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229365 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229368 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:22.331563 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229371 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:22.305909 ip-10-0-135-154 systemd[1]: Started Kubernetes Kubelet. Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229374 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229377 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229380 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229383 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229386 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229389 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229392 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229395 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229397 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229400 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229403 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229405 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229408 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229410 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229413 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229416 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229420 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229423 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229425 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:22.332580 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229429 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229432 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229434 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229437 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229440 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229444 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229448 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229451 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229454 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229457 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229460 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229463 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229467 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229470 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229473 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229476 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229479 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229481 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229484 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:22.333327 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229487 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229490 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229493 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229496 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229498 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229501 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229504 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229506 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229509 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229511 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229514 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229518 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229521 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229523 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229527 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229531 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229534 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229537 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229539 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229542 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:22.333977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229544 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229547 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229550 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229553 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229555 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229559 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229561 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229564 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229567 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229569 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229572 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229574 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229578 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229580 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229583 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229586 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229588 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229591 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229594 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229597 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:22.334697 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229600 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229602 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229605 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229607 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229610 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229612 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229615 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229618 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229620 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229623 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229625 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229628 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.229631 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230352 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230359 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230362 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230365 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230368 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230371 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230374 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:22.335336 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230376 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230379 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230382 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230385 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230388 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230390 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230393 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230395 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230398 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230401 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230403 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230406 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230410 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230412 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230415 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230417 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230420 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230423 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230426 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230428 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:22.336070 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230431 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230433 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230436 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230439 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230442 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230445 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230448 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230450 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230453 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230456 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230458 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230461 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230464 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230467 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230470 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230472 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230476 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230480 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230484 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:22.337370 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230486 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230489 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230491 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230494 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230497 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230499 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230502 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230505 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230508 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230512 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230516 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230520 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230522 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230525 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230528 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230532 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230534 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230537 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230540 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:22.338248 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230542 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230547 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230550 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230553 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230556 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230559 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230562 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230565 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230568 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230570 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230573 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230576 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230579 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230581 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230587 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230590 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230592 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230595 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230597 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:22.579822 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230600 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230602 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230605 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230608 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230611 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230614 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230616 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230619 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230621 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230624 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230626 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230628 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230631 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230633 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.230638 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231001 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231005 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231008 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231011 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231014 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:22.600203 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231016 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231019 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231024 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231028 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231031 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231034 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231037 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231044 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231047 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231050 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231052 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231055 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231058 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231060 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231063 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231065 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231068 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231070 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231073 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231075 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:22.600860 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231078 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231080 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231083 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231085 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231088 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231106 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231111 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231113 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231116 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231119 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231122 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231124 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231127 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231130 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231133 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231135 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231138 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231140 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231142 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231147 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:22.601554 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231150 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231152 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231155 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231158 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231160 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231163 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231166 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231168 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231171 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231174 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231177 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231180 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231182 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231184 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231187 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231190 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231192 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231194 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231198 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:22.602180 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231201 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231203 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231206 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231209 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231213 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231216 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231220 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231224 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231226 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231229 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231232 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231234 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231238 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231241 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231244 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231246 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231249 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231252 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231254 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:22.602797 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231257 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231260 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231262 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231265 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231267 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231270 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231272 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231275 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231277 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231280 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231282 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231285 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231289 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231291 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231294 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231297 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231373 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231381 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231388 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231393 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231397 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 03 10:05:22.603352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231400 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231404 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231409 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231412 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231416 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231419 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231422 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231425 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231428 2577 flags.go:64] FLAG: --cgroup-root="" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231431 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231434 2577 flags.go:64] FLAG: --client-ca-file="" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231437 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231440 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231444 2577 flags.go:64] FLAG: --cluster-domain="" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231447 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231450 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231453 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231456 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231460 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231463 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231466 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231470 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231472 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231478 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 03 10:05:22.603986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231481 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231484 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231487 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231491 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231494 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231496 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231499 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231502 2577 flags.go:64] FLAG: --enable-server="true" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231506 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231510 2577 flags.go:64] FLAG: --event-burst="100" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231514 2577 flags.go:64] FLAG: --event-qps="50" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231516 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231522 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231525 2577 flags.go:64] FLAG: --eviction-hard="" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231529 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231532 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231535 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231538 2577 flags.go:64] FLAG: --eviction-soft="" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231541 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231543 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231546 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231549 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231552 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231555 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231558 2577 flags.go:64] FLAG: --feature-gates="" Jul 03 10:05:22.604648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231562 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231565 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231568 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231571 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231574 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231577 2577 flags.go:64] FLAG: --help="false" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231581 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231585 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231588 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231591 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231594 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231598 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231601 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231604 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231606 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231609 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231613 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231617 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231620 2577 flags.go:64] FLAG: --kube-reserved="" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231624 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231627 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231630 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231632 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231635 2577 flags.go:64] FLAG: --lock-file="" Jul 03 10:05:22.605446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231638 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231641 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231644 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231649 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231652 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231655 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231657 2577 flags.go:64] FLAG: --logging-format="text" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231660 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231663 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231666 2577 flags.go:64] FLAG: --manifest-url="" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231669 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231674 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231677 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231680 2577 flags.go:64] FLAG: --max-pods="110" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231685 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231688 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231691 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231693 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231696 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231699 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231702 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231709 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231712 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231715 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 03 10:05:22.606104 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231718 2577 flags.go:64] FLAG: --pod-cidr="" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231722 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231727 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231730 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231735 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231738 2577 flags.go:64] FLAG: --port="10250" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231741 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231744 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-020e53aa29c3fc548" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231747 2577 flags.go:64] FLAG: --qos-reserved="" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231750 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231753 2577 flags.go:64] FLAG: --register-node="true" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231756 2577 flags.go:64] FLAG: --register-with-taints="" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231759 2577 flags.go:64] FLAG: --registry-burst="10" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231762 2577 flags.go:64] FLAG: --registry-qps="5" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231765 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231768 2577 flags.go:64] FLAG: --reserved-memory="" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231771 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231774 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231777 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231780 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231783 2577 flags.go:64] FLAG: --runonce="false" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231786 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231790 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231793 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231796 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 03 10:05:22.606814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231798 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231801 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231804 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231807 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231810 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231813 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231816 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231818 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231821 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231824 2577 flags.go:64] FLAG: --system-cgroups="" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231827 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231834 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231837 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231839 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231844 2577 flags.go:64] FLAG: --tls-min-version="" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231846 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231849 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231852 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231855 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231858 2577 flags.go:64] FLAG: --v="2" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231862 2577 flags.go:64] FLAG: --version="false" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231866 2577 flags.go:64] FLAG: --vmodule="" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231869 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.231872 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231987 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:22.607595 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231991 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231994 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231996 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.231999 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232003 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232006 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232009 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232012 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232014 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232017 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232019 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232022 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232024 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232027 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232030 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232032 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232035 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232038 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232042 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232044 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:22.608342 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232047 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232050 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232052 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232055 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232057 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232060 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232062 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232065 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232068 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232072 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232075 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232078 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232081 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232083 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232086 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232089 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232106 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232108 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232111 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232114 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:22.608907 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232117 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232119 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232122 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232125 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232128 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232130 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232133 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232135 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232138 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232141 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232145 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232148 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232151 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232153 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232156 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232159 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232161 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232164 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232168 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:22.609509 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232171 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232174 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232177 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232180 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232183 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232186 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232189 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232192 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232195 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232199 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232202 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232204 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232207 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232210 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232214 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232217 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232220 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232222 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232225 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:22.610126 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232227 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232230 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232233 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232235 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232239 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232242 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232245 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232248 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232250 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232253 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232255 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232258 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232260 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232263 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232266 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232268 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232271 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232273 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232276 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:22.610757 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.232279 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.232287 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.239677 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.239693 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.240642 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.240650 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240706 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240711 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240715 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240718 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240721 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240724 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240727 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240730 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240733 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:22.611390 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240736 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240738 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240742 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240745 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240747 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240750 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240753 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240755 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240758 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240760 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240763 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240765 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240768 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240770 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240773 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240776 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240778 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240782 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240787 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:22.612034 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240790 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240794 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240796 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240799 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240801 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240804 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240807 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240809 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240812 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240814 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240817 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240819 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240821 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240824 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240826 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240829 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240832 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240835 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240837 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240840 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:22.613363 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240843 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240846 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240848 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240851 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240853 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240856 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240858 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240861 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240865 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240869 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240872 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240875 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240878 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240881 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240885 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240888 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240892 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240894 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240898 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:22.613955 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240901 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240903 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240906 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240909 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240911 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240914 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240916 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240919 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240921 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240924 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240927 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240930 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240933 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240940 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240943 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240945 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240948 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240950 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240953 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240955 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:22.880631 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240958 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240960 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240963 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240965 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240968 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240971 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240976 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240979 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240982 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240985 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240987 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.240990 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.240996 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241149 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241156 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:22.881295 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241159 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241162 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241164 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241167 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241169 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241172 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241175 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241178 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241180 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241183 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241187 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241189 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241193 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241197 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241200 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241203 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241205 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241208 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241210 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:22.881738 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241213 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241215 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241218 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241220 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241223 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241227 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241229 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241233 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241235 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241238 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241241 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241244 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241246 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241249 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241251 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241254 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241256 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241259 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241261 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241264 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:22.882318 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241266 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241269 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241271 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241275 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241278 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241280 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241283 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241285 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241288 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241290 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241293 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241296 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241298 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241301 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241303 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241306 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241308 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241313 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241323 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241327 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:22.882869 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241331 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241334 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241337 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241340 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241343 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241345 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241348 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241350 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241353 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241355 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241358 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241361 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241364 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241366 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241369 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241375 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241378 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241381 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241383 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:22.883464 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241386 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241389 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241391 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241394 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241396 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241399 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241401 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241404 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241406 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241409 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241413 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241415 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241418 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241421 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241427 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241430 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241432 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241435 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:22.883977 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:22.241437 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.241442 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.241556 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.243773 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.245285 2577 server.go:1019] "Starting client certificate rotation" Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.245386 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.246382 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.247836 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.256262 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.264755 2577 log.go:25] "Validated CRI v1 runtime API" Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.268515 2577 log.go:25] "Validated CRI v1 image API" Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.269448 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 03 10:05:22.884557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.272555 2577 fs.go:135] Filesystem UUIDs: map[3353b23c-1a83-4534-9bd9-c58c2068b0c0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f9bd07d9-1fc8-4c04-9c2a-4456d6e6792a:/dev/nvme0n1p3] Jul 03 10:05:22.884919 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.272582 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 03 10:05:22.884919 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.275382 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.278072 2577 manager.go:216] Machine: {Timestamp:2026-07-03 10:05:22.276554098 +0000 UTC m=+0.342629281 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090792 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27322f573d5f4a12f267ecddd7265e SystemUUID:ec27322f-573d-5f4a-12f2-67ecddd7265e BootID:1f9881bf-b93e-47ad-a870-29b1034bb559 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ea:69:ad:c1:b7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ea:69:ad:c1:b7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:88:1f:4a:6e:55 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.278195 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.278322 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.281793 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.281818 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-154.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.282587 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.282597 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.282606 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.284105 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.285438 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.288588 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.288705 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.288733 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.288742 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.288777 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.288791 2577 kubelet.go:398] "Adding apiserver pod source" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.288800 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 03 10:05:22.885085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.289299 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.289333 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.292166 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.294600 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.294636 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.295946 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.295965 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.295976 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.295981 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.295991 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.295997 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.296005 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.296011 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.296021 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.296027 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.296035 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.296041 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.297264 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.297275 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.300614 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-154.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.300592 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.300687 2577 server.go:1268] "Started kubelet" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.300742 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.300746 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.304860 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.300680 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-154.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.306903 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.307675 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.311255 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.314127 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.314812 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.315534 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.315545 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.315628 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.315676 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.315683 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.316325 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.317606 2577 factory.go:55] Registering systemd factory Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.317618 2577 factory.go:223] Registration of the systemd container factory successfully Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.317829 2577 factory.go:153] Registering CRI-O factory Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.317846 2577 factory.go:223] Registration of the crio container factory successfully Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.317898 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.317923 2577 factory.go:103] Registering Raw factory Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.317939 2577 manager.go:1196] Started watching for new ooms in manager Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.318407 2577 manager.go:349] Starting recovery of all containers Jul 03 10:05:22.885733 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.322008 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-154.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.322074 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.322826 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.322212 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-154.ec2.internal.18bebf8ebf456671 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-154.ec2.internal,UID:ip-10-0-135-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-154.ec2.internal,},FirstTimestamp:2026-07-03 10:05:22.300552817 +0000 UTC m=+0.366628000,LastTimestamp:2026-07-03 10:05:22.300552817 +0000 UTC m=+0.366628000,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-154.ec2.internal,}" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.327943 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tb7s4" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.328610 2577 manager.go:354] Recovery completed Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.333475 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tb7s4" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.333948 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.336019 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.336054 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.336068 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.336658 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.336666 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.336675 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.336681 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.337555 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-154.ec2.internal.18bebf8ec162bf78 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-154.ec2.internal,UID:ip-10-0-135-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-154.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-154.ec2.internal,},FirstTimestamp:2026-07-03 10:05:22.336030584 +0000 UTC m=+0.402105770,LastTimestamp:2026-07-03 10:05:22.336030584 +0000 UTC m=+0.402105770,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-154.ec2.internal,}" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.337908 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.337931 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.337940 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.337948 2577 policy_none.go:49] "None policy: Start" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.337961 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.337981 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.339123 2577 policy_none.go:47] "Start" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.384416 2577 manager.go:337] "Starting Device Plugin manager" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.384448 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.384461 2577 server.go:87] "Starting device plugin registration server" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.384709 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.384846 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.384962 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.385051 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.385058 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.385591 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.385641 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.465232 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 03 10:05:22.886927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.465274 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.465299 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.465354 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.468061 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.485454 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.486392 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.486424 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.486435 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.486467 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.496670 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.496690 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-154.ec2.internal\": node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.525387 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.566136 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal"] Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.566214 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.567276 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.567307 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.567317 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.568718 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.568842 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.568873 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.569448 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.569464 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.569473 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.569482 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.569487 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.569498 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.570742 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.570766 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.571456 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.571484 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.571494 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.586719 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-154.ec2.internal\" not found" node="ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.590710 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-154.ec2.internal\" not found" node="ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.617703 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/543c32978d2cab93351cc69b95518e92-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"543c32978d2cab93351cc69b95518e92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.887939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.617730 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/543c32978d2cab93351cc69b95518e92-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"543c32978d2cab93351cc69b95518e92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.888860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.617760 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40608af7edd160b369e43165ab923d25-config\") pod \"kube-apiserver-proxy-ip-10-0-135-154.ec2.internal\" (UID: \"40608af7edd160b369e43165ab923d25\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.888860 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.626337 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:22.888860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.718448 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/543c32978d2cab93351cc69b95518e92-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"543c32978d2cab93351cc69b95518e92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.888860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.718478 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/543c32978d2cab93351cc69b95518e92-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"543c32978d2cab93351cc69b95518e92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.888860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.718495 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40608af7edd160b369e43165ab923d25-config\") pod \"kube-apiserver-proxy-ip-10-0-135-154.ec2.internal\" (UID: \"40608af7edd160b369e43165ab923d25\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.888860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.718539 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40608af7edd160b369e43165ab923d25-config\") pod \"kube-apiserver-proxy-ip-10-0-135-154.ec2.internal\" (UID: \"40608af7edd160b369e43165ab923d25\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.888860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.718559 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/543c32978d2cab93351cc69b95518e92-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"543c32978d2cab93351cc69b95518e92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.888860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.718587 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/543c32978d2cab93351cc69b95518e92-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"543c32978d2cab93351cc69b95518e92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" Jul 03 10:05:22.888860 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.729544 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:22.888860 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.830170 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:23.130299 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.889498 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" Jul 03 10:05:23.130299 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:22.896643 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" Jul 03 10:05:23.130299 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:22.930785 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:23.130299 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:23.031748 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:23.349371 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:23.132685 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:23.349371 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:23.233078 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:23.349371 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.244673 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 03 10:05:23.349371 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.244848 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 03 10:05:23.349371 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:23.334059 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found" Jul 03 10:05:23.349371 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.334901 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 10:05:23.349371 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.337390 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-02 10:00:22 +0000 UTC" deadline="2027-12-22 18:46:57.222433427 +0000 UTC" Jul 03 10:05:23.349371 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.337409 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12896h41m33.885027007s" Jul 03 10:05:23.554502 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.416586 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" Jul 03 10:05:23.554502 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.433681 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 10:05:23.554502 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.435671 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" Jul 03 10:05:23.554502 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.445893 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 10:05:23.554502 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.515414 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 10:05:23.752936 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:23.645590 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod543c32978d2cab93351cc69b95518e92.slice/crio-c169493ed7ea4e8087520b2038802e91f2010f0e7ed3d398b8825c32aeec2fbc WatchSource:0}: Error finding container c169493ed7ea4e8087520b2038802e91f2010f0e7ed3d398b8825c32aeec2fbc: Status 404 returned error can't find the container with id c169493ed7ea4e8087520b2038802e91f2010f0e7ed3d398b8825c32aeec2fbc Jul 03 10:05:23.752936 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:23.645765 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40608af7edd160b369e43165ab923d25.slice/crio-32ec71bfd94ef94a8b1ec339373df363aa94a820076a8eeee95b4c9e0aa9c56f WatchSource:0}: Error finding container 32ec71bfd94ef94a8b1ec339373df363aa94a820076a8eeee95b4c9e0aa9c56f: Status 404 returned error can't find the container with id 32ec71bfd94ef94a8b1ec339373df363aa94a820076a8eeee95b4c9e0aa9c56f Jul 03 10:05:23.752936 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.654087 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:05:23.752936 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:23.655571 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 10:05:24.309807 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.303868 2577 apiserver.go:52] "Watching apiserver" Jul 03 10:05:24.315549 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.315230 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 03 10:05:24.315549 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.315364 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 03 10:05:24.318398 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.317519 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-rrn2q","kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh","openshift-dns/node-resolver-kzm94","openshift-multus/multus-additional-cni-plugins-gcpsk","openshift-multus/network-metrics-daemon-2rjc5","openshift-network-diagnostics/network-check-target-hkgr6","openshift-ovn-kubernetes/ovnkube-node-hb8xm","kube-system/global-pull-secret-syncer-vz6gj","openshift-cluster-node-tuning-operator/tuned-pq999","openshift-image-registry/node-ca-bm5md","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal","openshift-multus/multus-g55zz","openshift-network-operator/iptables-alerter-fwkpz"] Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.320052 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.320136 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.322175 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.324420 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.325067 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.325123 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.325331 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.325432 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-b84kz\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.325337 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.326632 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.327192 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.327202 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w72hs\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.327385 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.327407 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.329425 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.329797 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4spk7\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.329828 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.330054 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.332072 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.332157 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.332074 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.332561 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.332855 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9fbrg\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.332925 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.333920 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.334002 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.336838 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.339927 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.340134 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2hxjr\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.340330 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.341288 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.344405 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:05:24.403031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.344520 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.344983 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rt49l\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.345396 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.345528 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.346930 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.349427 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.351235 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.351548 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lrljg\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.351955 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.352206 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.352444 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.352658 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tmv84\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.353263 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.354407 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.354655 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h8jtr\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.354725 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.354824 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.354838 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.354856 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vrhs5" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.355195 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.355981 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 03 10:05:24.405784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.363408 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vrhs5" Jul 03 10:05:24.417946 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.416950 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430663 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-host\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430699 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-node-log\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430723 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-cni-bin\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430787 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25536018-295a-44e5-b396-44d6f533d2b2-ovnkube-config\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430819 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-system-cni-dir\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430874 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45b940e2-1138-4ed4-95a8-db85ef6328b2-iptables-alerter-script\") pod \"iptables-alerter-fwkpz\" (UID: \"45b940e2-1138-4ed4-95a8-db85ef6328b2\") " pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430915 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-modprobe-d\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/108d4f92-ca93-47d8-a45f-ba7088301bfa-kubelet-config\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430962 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8cd9\" (UniqueName: \"kubernetes.io/projected/25536018-295a-44e5-b396-44d6f533d2b2-kube-api-access-g8cd9\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430977 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-os-release\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.430994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-socket-dir-parent\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431012 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431075 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-sysconfig\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431104 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-kubernetes\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.494142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431131 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-sysctl-conf\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431151 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-var-lib-kubelet\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25536018-295a-44e5-b396-44d6f533d2b2-ovn-node-metrics-cert\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431188 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/108d4f92-ca93-47d8-a45f-ba7088301bfa-dbus\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431224 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-var-lib-openvswitch\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431248 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431267 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqkl8\" (UniqueName: \"kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8\") pod \"network-check-target-hkgr6\" (UID: \"12d45bd8-f937-4c62-98a4-63a18defb862\") " pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431288 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-tmp\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431321 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-var-lib-cni-bin\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-var-lib-kubelet\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431381 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-kubelet\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431409 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-run-openvswitch\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431433 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-cni-binary-copy\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ngvl\" (UniqueName: \"kubernetes.io/projected/45b940e2-1138-4ed4-95a8-db85ef6328b2-kube-api-access-6ngvl\") pod \"iptables-alerter-fwkpz\" (UID: \"45b940e2-1138-4ed4-95a8-db85ef6328b2\") " pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431478 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-system-cni-dir\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431499 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-var-lib-cni-multus\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.495553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431522 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdzmz\" (UniqueName: \"kubernetes.io/projected/01500b78-d50b-467c-bf74-f4b3e310e159-kube-api-access-cdzmz\") pod \"node-ca-bm5md\" (UID: \"01500b78-d50b-467c-bf74-f4b3e310e159\") " pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431543 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-log-socket\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431564 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25536018-295a-44e5-b396-44d6f533d2b2-env-overrides\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431584 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25536018-295a-44e5-b396-44d6f533d2b2-ovnkube-script-lib\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431607 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-etc-selinux\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431631 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-os-release\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431654 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431674 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-lib-modules\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431697 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431718 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75626273-af77-4838-99cc-2d3b274069d4-tmp-dir\") pod \"node-resolver-kzm94\" (UID: \"75626273-af77-4838-99cc-2d3b274069d4\") " pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431742 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-cnibin\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431764 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89tw9\" (UniqueName: \"kubernetes.io/projected/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-kube-api-access-89tw9\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431788 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef4101c3-8dd9-450e-96d1-e7e8159cf062-agent-certs\") pod \"konnectivity-agent-rrn2q\" (UID: \"ef4101c3-8dd9-450e-96d1-e7e8159cf062\") " pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431809 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-run-k8s-cni-cncf-io\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-daemon-config\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431852 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfgv5\" (UniqueName: \"kubernetes.io/projected/57e21cd6-3595-4064-95a5-2b22c0824be6-kube-api-access-lfgv5\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.496516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431887 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-systemd-units\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431906 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-device-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431931 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45b940e2-1138-4ed4-95a8-db85ef6328b2-host-slash\") pod \"iptables-alerter-fwkpz\" (UID: \"45b940e2-1138-4ed4-95a8-db85ef6328b2\") " pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431951 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-run\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431973 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.431995 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6g6p\" (UniqueName: \"kubernetes.io/projected/ad84ff02-cefd-4c8e-b897-8a97a3727b79-kube-api-access-v6g6p\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432017 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-conf-dir\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432038 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-etc-kubernetes\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-run-netns\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-socket-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6cq7\" (UniqueName: \"kubernetes.io/projected/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-kube-api-access-g6cq7\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432177 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/75626273-af77-4838-99cc-2d3b274069d4-hosts-file\") pod \"node-resolver-kzm94\" (UID: \"75626273-af77-4838-99cc-2d3b274069d4\") " pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432198 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef4101c3-8dd9-450e-96d1-e7e8159cf062-konnectivity-ca\") pod \"konnectivity-agent-rrn2q\" (UID: \"ef4101c3-8dd9-450e-96d1-e7e8159cf062\") " pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432219 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-run-netns\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432242 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-hostroot\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.500704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432265 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-slash\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432287 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-run-ovn\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432307 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-registration-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432329 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-systemd\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432349 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-sys\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432369 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-tuned\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432392 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-cni-dir\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01500b78-d50b-467c-bf74-f4b3e310e159-host\") pod \"node-ca-bm5md\" (UID: \"01500b78-d50b-467c-bf74-f4b3e310e159\") " pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432433 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-sys-fs\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432455 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95p9w\" (UniqueName: \"kubernetes.io/projected/75626273-af77-4838-99cc-2d3b274069d4-kube-api-access-95p9w\") pod \"node-resolver-kzm94\" (UID: \"75626273-af77-4838-99cc-2d3b274069d4\") " pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432479 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432499 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-cnibin\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432520 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/57e21cd6-3595-4064-95a5-2b22c0824be6-cni-binary-copy\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432542 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-etc-openvswitch\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432563 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zbqv\" (UniqueName: \"kubernetes.io/projected/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-kube-api-access-8zbqv\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432584 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-run-systemd\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432606 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-run-multus-certs\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.501550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432626 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-cni-netd\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.503481 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432656 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/01500b78-d50b-467c-bf74-f4b3e310e159-serviceca\") pod \"node-ca-bm5md\" (UID: \"01500b78-d50b-467c-bf74-f4b3e310e159\") " pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.503481 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.432677 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-sysctl-d\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.503481 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.473722 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" event={"ID":"543c32978d2cab93351cc69b95518e92","Type":"ContainerStarted","Data":"c169493ed7ea4e8087520b2038802e91f2010f0e7ed3d398b8825c32aeec2fbc"} Jul 03 10:05:24.503481 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.474459 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" event={"ID":"40608af7edd160b369e43165ab923d25","Type":"ContainerStarted","Data":"32ec71bfd94ef94a8b1ec339373df363aa94a820076a8eeee95b4c9e0aa9c56f"} Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534478 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-run-netns\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534521 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-socket-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6cq7\" (UniqueName: \"kubernetes.io/projected/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-kube-api-access-g6cq7\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/75626273-af77-4838-99cc-2d3b274069d4-hosts-file\") pod \"node-resolver-kzm94\" (UID: \"75626273-af77-4838-99cc-2d3b274069d4\") " pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef4101c3-8dd9-450e-96d1-e7e8159cf062-konnectivity-ca\") pod \"konnectivity-agent-rrn2q\" (UID: \"ef4101c3-8dd9-450e-96d1-e7e8159cf062\") " pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534644 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-run-netns\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534667 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-hostroot\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534690 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-slash\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-run-ovn\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534736 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-registration-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534758 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-systemd\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534780 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-sys\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534804 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-tuned\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534826 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-cni-dir\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534848 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01500b78-d50b-467c-bf74-f4b3e310e159-host\") pod \"node-ca-bm5md\" (UID: \"01500b78-d50b-467c-bf74-f4b3e310e159\") " pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.570574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534871 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-sys-fs\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534895 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95p9w\" (UniqueName: \"kubernetes.io/projected/75626273-af77-4838-99cc-2d3b274069d4-kube-api-access-95p9w\") pod \"node-resolver-kzm94\" (UID: \"75626273-af77-4838-99cc-2d3b274069d4\") " pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534944 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534967 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-cnibin\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.534990 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/57e21cd6-3595-4064-95a5-2b22c0824be6-cni-binary-copy\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535014 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-etc-openvswitch\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535032 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zbqv\" (UniqueName: \"kubernetes.io/projected/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-kube-api-access-8zbqv\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-run-systemd\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535075 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-run-multus-certs\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535089 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-cni-netd\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535125 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/01500b78-d50b-467c-bf74-f4b3e310e159-serviceca\") pod \"node-ca-bm5md\" (UID: \"01500b78-d50b-467c-bf74-f4b3e310e159\") " pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-sysctl-d\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535153 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-host\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535177 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-node-log\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535190 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-cni-bin\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535216 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25536018-295a-44e5-b396-44d6f533d2b2-ovnkube-config\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535235 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-system-cni-dir\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.572761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535250 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535268 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45b940e2-1138-4ed4-95a8-db85ef6328b2-iptables-alerter-script\") pod \"iptables-alerter-fwkpz\" (UID: \"45b940e2-1138-4ed4-95a8-db85ef6328b2\") " pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535283 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-modprobe-d\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535299 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/108d4f92-ca93-47d8-a45f-ba7088301bfa-kubelet-config\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8cd9\" (UniqueName: \"kubernetes.io/projected/25536018-295a-44e5-b396-44d6f533d2b2-kube-api-access-g8cd9\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535328 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-os-release\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535359 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-socket-dir-parent\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535375 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-sysconfig\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535412 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-kubernetes\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-sysctl-conf\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535452 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-var-lib-kubelet\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535476 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25536018-295a-44e5-b396-44d6f533d2b2-ovn-node-metrics-cert\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535497 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/108d4f92-ca93-47d8-a45f-ba7088301bfa-dbus\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535518 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-var-lib-openvswitch\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535540 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535563 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkl8\" (UniqueName: \"kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8\") pod \"network-check-target-hkgr6\" (UID: \"12d45bd8-f937-4c62-98a4-63a18defb862\") " pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:24.574320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535585 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-tmp\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535608 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-var-lib-cni-bin\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535633 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-var-lib-kubelet\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535656 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-kubelet\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535679 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-run-openvswitch\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535698 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-cni-binary-copy\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535714 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ngvl\" (UniqueName: \"kubernetes.io/projected/45b940e2-1138-4ed4-95a8-db85ef6328b2-kube-api-access-6ngvl\") pod \"iptables-alerter-fwkpz\" (UID: \"45b940e2-1138-4ed4-95a8-db85ef6328b2\") " pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535729 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-system-cni-dir\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535745 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-var-lib-cni-multus\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdzmz\" (UniqueName: \"kubernetes.io/projected/01500b78-d50b-467c-bf74-f4b3e310e159-kube-api-access-cdzmz\") pod \"node-ca-bm5md\" (UID: \"01500b78-d50b-467c-bf74-f4b3e310e159\") " pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-log-socket\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535792 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25536018-295a-44e5-b396-44d6f533d2b2-env-overrides\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25536018-295a-44e5-b396-44d6f533d2b2-ovnkube-script-lib\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-etc-selinux\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535869 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-os-release\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535886 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535903 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-lib-modules\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.577544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535922 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535945 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75626273-af77-4838-99cc-2d3b274069d4-tmp-dir\") pod \"node-resolver-kzm94\" (UID: \"75626273-af77-4838-99cc-2d3b274069d4\") " pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535961 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-cnibin\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535977 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89tw9\" (UniqueName: \"kubernetes.io/projected/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-kube-api-access-89tw9\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.535993 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef4101c3-8dd9-450e-96d1-e7e8159cf062-agent-certs\") pod \"konnectivity-agent-rrn2q\" (UID: \"ef4101c3-8dd9-450e-96d1-e7e8159cf062\") " pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536009 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-run-k8s-cni-cncf-io\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-daemon-config\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfgv5\" (UniqueName: \"kubernetes.io/projected/57e21cd6-3595-4064-95a5-2b22c0824be6-kube-api-access-lfgv5\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536062 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-systemd-units\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536077 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-device-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536110 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45b940e2-1138-4ed4-95a8-db85ef6328b2-host-slash\") pod \"iptables-alerter-fwkpz\" (UID: \"45b940e2-1138-4ed4-95a8-db85ef6328b2\") " pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-run\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536143 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536162 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6g6p\" (UniqueName: \"kubernetes.io/projected/ad84ff02-cefd-4c8e-b897-8a97a3727b79-kube-api-access-v6g6p\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536178 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-conf-dir\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536200 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-etc-kubernetes\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536282 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-etc-kubernetes\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.578414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536320 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-run-netns\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536385 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536476 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-socket-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.536718 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/75626273-af77-4838-99cc-2d3b274069d4-hosts-file\") pod \"node-resolver-kzm94\" (UID: \"75626273-af77-4838-99cc-2d3b274069d4\") " pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537170 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef4101c3-8dd9-450e-96d1-e7e8159cf062-konnectivity-ca\") pod \"konnectivity-agent-rrn2q\" (UID: \"ef4101c3-8dd9-450e-96d1-e7e8159cf062\") " pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537230 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-run-netns\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537259 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-hostroot\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537284 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-slash\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537309 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-run-ovn\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537360 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-registration-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537400 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-systemd\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537434 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-sys\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537662 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537668 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-var-lib-openvswitch\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537851 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-cni-dir\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537909 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01500b78-d50b-467c-bf74-f4b3e310e159-host\") pod \"node-ca-bm5md\" (UID: \"01500b78-d50b-467c-bf74-f4b3e310e159\") " pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.537980 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-sys-fs\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.539432 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-cnibin\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.636337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.540002 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/57e21cd6-3595-4064-95a5-2b22c0824be6-cni-binary-copy\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.540036 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.540500 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-modprobe-d\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.540575 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/108d4f92-ca93-47d8-a45f-ba7088301bfa-kubelet-config\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.540626 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-os-release\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.540695 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.540919 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-host\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.540933 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-node-log\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.540968 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-run-systemd\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541023 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-cni-bin\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541041 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-run-multus-certs\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541049 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-var-lib-kubelet\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541076 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-var-lib-cni-bin\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541084 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-cni-netd\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541271 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-sysctl-d\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541500 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-tuned\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541663 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-kubernetes\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541823 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-os-release\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.637635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541851 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25536018-295a-44e5-b396-44d6f533d2b2-ovnkube-config\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.541894 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-socket-dir-parent\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.541997 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542025 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542209 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-lib-modules\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542271 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542304 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/01500b78-d50b-467c-bf74-f4b3e310e159-serviceca\") pod \"node-ca-bm5md\" (UID: \"01500b78-d50b-467c-bf74-f4b3e310e159\") " pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.542360 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret podName:108d4f92-ca93-47d8-a45f-ba7088301bfa nodeName:}" failed. No retries permitted until 2026-07-03 10:05:25.042338478 +0000 UTC m=+3.108413665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret") pod "global-pull-secret-syncer-vz6gj" (UID: "108d4f92-ca93-47d8-a45f-ba7088301bfa") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542425 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-sysconfig\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542498 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-system-cni-dir\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542543 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75626273-af77-4838-99cc-2d3b274069d4-tmp-dir\") pod \"node-resolver-kzm94\" (UID: \"75626273-af77-4838-99cc-2d3b274069d4\") " pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542606 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-cnibin\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542746 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-var-lib-kubelet\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542843 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-etc-sysctl-conf\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542897 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-var-lib-cni-multus\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.542958 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-run-openvswitch\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.543156 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.639415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.543293 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-etc-openvswitch\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.543463 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-cni-binary-copy\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.543531 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-system-cni-dir\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.543555 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-host-kubelet\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.543710 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.543752 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25536018-295a-44e5-b396-44d6f533d2b2-env-overrides\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.543780 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-log-socket\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.543951 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs podName:ad84ff02-cefd-4c8e-b897-8a97a3727b79 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:25.043933072 +0000 UTC m=+3.110008255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs") pod "network-metrics-daemon-2rjc5" (UID: "ad84ff02-cefd-4c8e-b897-8a97a3727b79") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.544025 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25536018-295a-44e5-b396-44d6f533d2b2-systemd-units\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.544110 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-device-dir\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.544168 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-host-run-k8s-cni-cncf-io\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.544058 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25536018-295a-44e5-b396-44d6f533d2b2-ovnkube-script-lib\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.544263 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45b940e2-1138-4ed4-95a8-db85ef6328b2-host-slash\") pod \"iptables-alerter-fwkpz\" (UID: \"45b940e2-1138-4ed4-95a8-db85ef6328b2\") " pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.544283 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/108d4f92-ca93-47d8-a45f-ba7088301bfa-dbus\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.544363 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-etc-selinux\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.544364 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-run\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.544434 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-conf-dir\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.641489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.544953 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/57e21cd6-3595-4064-95a5-2b22c0824be6-multus-daemon-config\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.545500 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45b940e2-1138-4ed4-95a8-db85ef6328b2-iptables-alerter-script\") pod \"iptables-alerter-fwkpz\" (UID: \"45b940e2-1138-4ed4-95a8-db85ef6328b2\") " pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.547954 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.547982 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.547997 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cqkl8 for pod openshift-network-diagnostics/network-check-target-hkgr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:24.548063 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8 podName:12d45bd8-f937-4c62-98a4-63a18defb862 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:25.048047098 +0000 UTC m=+3.114122292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cqkl8" (UniqueName: "kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8") pod "network-check-target-hkgr6" (UID: "12d45bd8-f937-4c62-98a4-63a18defb862") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.551215 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfgv5\" (UniqueName: \"kubernetes.io/projected/57e21cd6-3595-4064-95a5-2b22c0824be6-kube-api-access-lfgv5\") pod \"multus-g55zz\" (UID: \"57e21cd6-3595-4064-95a5-2b22c0824be6\") " pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.551521 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95p9w\" (UniqueName: \"kubernetes.io/projected/75626273-af77-4838-99cc-2d3b274069d4-kube-api-access-95p9w\") pod \"node-resolver-kzm94\" (UID: \"75626273-af77-4838-99cc-2d3b274069d4\") " pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.551599 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6cq7\" (UniqueName: \"kubernetes.io/projected/c0ae8a49-7ced-4ffa-8757-d5034a4b3752-kube-api-access-g6cq7\") pod \"aws-ebs-csi-driver-node-x9hnh\" (UID: \"c0ae8a49-7ced-4ffa-8757-d5034a4b3752\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.551947 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef4101c3-8dd9-450e-96d1-e7e8159cf062-agent-certs\") pod \"konnectivity-agent-rrn2q\" (UID: \"ef4101c3-8dd9-450e-96d1-e7e8159cf062\") " pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.552854 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25536018-295a-44e5-b396-44d6f533d2b2-ovn-node-metrics-cert\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.554984 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89tw9\" (UniqueName: \"kubernetes.io/projected/3a89dafb-6d42-40f3-b9d4-cb4a0c839af6-kube-api-access-89tw9\") pod \"multus-additional-cni-plugins-gcpsk\" (UID: \"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6\") " pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.557221 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdzmz\" (UniqueName: \"kubernetes.io/projected/01500b78-d50b-467c-bf74-f4b3e310e159-kube-api-access-cdzmz\") pod \"node-ca-bm5md\" (UID: \"01500b78-d50b-467c-bf74-f4b3e310e159\") " pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.557385 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zbqv\" (UniqueName: \"kubernetes.io/projected/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-kube-api-access-8zbqv\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.558315 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ngvl\" (UniqueName: \"kubernetes.io/projected/45b940e2-1138-4ed4-95a8-db85ef6328b2-kube-api-access-6ngvl\") pod \"iptables-alerter-fwkpz\" (UID: \"45b940e2-1138-4ed4-95a8-db85ef6328b2\") " pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.559418 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6g6p\" (UniqueName: \"kubernetes.io/projected/ad84ff02-cefd-4c8e-b897-8a97a3727b79-kube-api-access-v6g6p\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.559472 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8cd9\" (UniqueName: \"kubernetes.io/projected/25536018-295a-44e5-b396-44d6f533d2b2-kube-api-access-g8cd9\") pod \"ovnkube-node-hb8xm\" (UID: \"25536018-295a-44e5-b396-44d6f533d2b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.642561 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.559647 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8bd73b8-7b17-4f0b-a23b-d343da75dd3a-tmp\") pod \"tuned-pq999\" (UID: \"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a\") " pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.643930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.635425 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g55zz" Jul 03 10:05:24.646735 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:24.646144 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e21cd6_3595_4064_95a5_2b22c0824be6.slice/crio-0f816df7099563fa500f66ce512a26e72799f14b79e5bd1dd1f544e0fcb11f15 WatchSource:0}: Error finding container 0f816df7099563fa500f66ce512a26e72799f14b79e5bd1dd1f544e0fcb11f15: Status 404 returned error can't find the container with id 0f816df7099563fa500f66ce512a26e72799f14b79e5bd1dd1f544e0fcb11f15 Jul 03 10:05:24.647522 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.647502 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" Jul 03 10:05:24.657557 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:24.657501 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ae8a49_7ced_4ffa_8757_d5034a4b3752.slice/crio-7d0fe2c44c40d9cf520a9715f5c2d94b7561069bfa1084b73587378f2f95b9e2 WatchSource:0}: Error finding container 7d0fe2c44c40d9cf520a9715f5c2d94b7561069bfa1084b73587378f2f95b9e2: Status 404 returned error can't find the container with id 7d0fe2c44c40d9cf520a9715f5c2d94b7561069bfa1084b73587378f2f95b9e2 Jul 03 10:05:24.666273 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.665660 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kzm94" Jul 03 10:05:24.670745 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.670725 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" Jul 03 10:05:24.676704 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:24.676661 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75626273_af77_4838_99cc_2d3b274069d4.slice/crio-05c5aac59c4a69450abb1fb427b483e4255f96d31ae33994ed50539e4fd60d5f WatchSource:0}: Error finding container 05c5aac59c4a69450abb1fb427b483e4255f96d31ae33994ed50539e4fd60d5f: Status 404 returned error can't find the container with id 05c5aac59c4a69450abb1fb427b483e4255f96d31ae33994ed50539e4fd60d5f Jul 03 10:05:24.678190 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.678168 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:24.681275 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:24.681217 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a89dafb_6d42_40f3_b9d4_cb4a0c839af6.slice/crio-b19f4968d89fc248df5e9bad0515bacdaa0360e9a7845fc2cd454330f9b16fb5 WatchSource:0}: Error finding container b19f4968d89fc248df5e9bad0515bacdaa0360e9a7845fc2cd454330f9b16fb5: Status 404 returned error can't find the container with id b19f4968d89fc248df5e9bad0515bacdaa0360e9a7845fc2cd454330f9b16fb5 Jul 03 10:05:24.685408 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.685383 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 10:05:24.687812 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:24.687789 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef4101c3_8dd9_450e_96d1_e7e8159cf062.slice/crio-e431faaf28b2d46acb360810d72baa10fe4bafcb0696bde384cda5ec59188ff2 WatchSource:0}: Error finding container e431faaf28b2d46acb360810d72baa10fe4bafcb0696bde384cda5ec59188ff2: Status 404 returned error can't find the container with id e431faaf28b2d46acb360810d72baa10fe4bafcb0696bde384cda5ec59188ff2 Jul 03 10:05:24.696050 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.695936 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fwkpz" Jul 03 10:05:24.705147 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.704975 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pq999" Jul 03 10:05:24.707863 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:24.707843 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b940e2_1138_4ed4_95a8_db85ef6328b2.slice/crio-8421d02a435c75eb2d37489aac51c313ae05297e0105a463e6ca1f95c451bbed WatchSource:0}: Error finding container 8421d02a435c75eb2d37489aac51c313ae05297e0105a463e6ca1f95c451bbed: Status 404 returned error can't find the container with id 8421d02a435c75eb2d37489aac51c313ae05297e0105a463e6ca1f95c451bbed Jul 03 10:05:24.712590 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.712527 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bm5md" Jul 03 10:05:24.714084 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:24.714058 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8bd73b8_7b17_4f0b_a23b_d343da75dd3a.slice/crio-c1dd165cccd7b79d6523b48d5684269aec72eb17951d5a5c857895c94064278d WatchSource:0}: Error finding container c1dd165cccd7b79d6523b48d5684269aec72eb17951d5a5c857895c94064278d: Status 404 returned error can't find the container with id c1dd165cccd7b79d6523b48d5684269aec72eb17951d5a5c857895c94064278d Jul 03 10:05:24.719609 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:24.719585 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:24.727431 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:24.727401 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01500b78_d50b_467c_bf74_f4b3e310e159.slice/crio-3ef0bcc67952349723d5162a5aba2f82eba569999887df90ed1db253f0992ee2 WatchSource:0}: Error finding container 3ef0bcc67952349723d5162a5aba2f82eba569999887df90ed1db253f0992ee2: Status 404 returned error can't find the container with id 3ef0bcc67952349723d5162a5aba2f82eba569999887df90ed1db253f0992ee2 Jul 03 10:05:24.734808 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:24.734782 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25536018_295a_44e5_b396_44d6f533d2b2.slice/crio-909871783a9c6e4845545dbaf39e3baa8bfb24f315a7e1ab58e9ed9f5e0e8b5b WatchSource:0}: Error finding container 909871783a9c6e4845545dbaf39e3baa8bfb24f315a7e1ab58e9ed9f5e0e8b5b: Status 404 returned error can't find the container with id 909871783a9c6e4845545dbaf39e3baa8bfb24f315a7e1ab58e9ed9f5e0e8b5b Jul 03 10:05:25.140620 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.140575 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:25.140808 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.140639 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkl8\" (UniqueName: \"kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8\") pod \"network-check-target-hkgr6\" (UID: \"12d45bd8-f937-4c62-98a4-63a18defb862\") " pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:25.140808 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.140690 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:25.140808 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:25.140744 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:25.140808 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:25.140798 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:25.140808 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:25.140802 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:25.141050 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:25.140824 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:25.141050 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:25.140826 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret podName:108d4f92-ca93-47d8-a45f-ba7088301bfa nodeName:}" failed. No retries permitted until 2026-07-03 10:05:26.140806585 +0000 UTC m=+4.206881759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret") pod "global-pull-secret-syncer-vz6gj" (UID: "108d4f92-ca93-47d8-a45f-ba7088301bfa") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:25.141050 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:25.140834 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cqkl8 for pod openshift-network-diagnostics/network-check-target-hkgr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:25.141050 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:25.140847 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs podName:ad84ff02-cefd-4c8e-b897-8a97a3727b79 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:26.140834939 +0000 UTC m=+4.206910112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs") pod "network-metrics-daemon-2rjc5" (UID: "ad84ff02-cefd-4c8e-b897-8a97a3727b79") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:25.141050 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:25.140873 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8 podName:12d45bd8-f937-4c62-98a4-63a18defb862 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:26.140857404 +0000 UTC m=+4.206932593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqkl8" (UniqueName: "kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8") pod "network-check-target-hkgr6" (UID: "12d45bd8-f937-4c62-98a4-63a18defb862") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:25.366696 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.366653 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 10:00:24 +0000 UTC" deadline="2028-03-15 16:48:42.788407674 +0000 UTC" Jul 03 10:05:25.366696 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.366691 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14910h43m17.421721684s" Jul 03 10:05:25.465623 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.465550 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:25.466164 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:25.465664 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:25.478034 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.477998 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" event={"ID":"25536018-295a-44e5-b396-44d6f533d2b2","Type":"ContainerStarted","Data":"909871783a9c6e4845545dbaf39e3baa8bfb24f315a7e1ab58e9ed9f5e0e8b5b"} Jul 03 10:05:25.479268 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.479238 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bm5md" event={"ID":"01500b78-d50b-467c-bf74-f4b3e310e159","Type":"ContainerStarted","Data":"3ef0bcc67952349723d5162a5aba2f82eba569999887df90ed1db253f0992ee2"} Jul 03 10:05:25.480385 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.480351 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pq999" event={"ID":"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a","Type":"ContainerStarted","Data":"c1dd165cccd7b79d6523b48d5684269aec72eb17951d5a5c857895c94064278d"} Jul 03 10:05:25.481583 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.481551 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fwkpz" event={"ID":"45b940e2-1138-4ed4-95a8-db85ef6328b2","Type":"ContainerStarted","Data":"8421d02a435c75eb2d37489aac51c313ae05297e0105a463e6ca1f95c451bbed"} Jul 03 10:05:25.482721 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.482691 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kzm94" event={"ID":"75626273-af77-4838-99cc-2d3b274069d4","Type":"ContainerStarted","Data":"05c5aac59c4a69450abb1fb427b483e4255f96d31ae33994ed50539e4fd60d5f"} Jul 03 10:05:25.483843 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.483813 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" event={"ID":"c0ae8a49-7ced-4ffa-8757-d5034a4b3752","Type":"ContainerStarted","Data":"7d0fe2c44c40d9cf520a9715f5c2d94b7561069bfa1084b73587378f2f95b9e2"} Jul 03 10:05:25.484972 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.484937 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rrn2q" event={"ID":"ef4101c3-8dd9-450e-96d1-e7e8159cf062","Type":"ContainerStarted","Data":"e431faaf28b2d46acb360810d72baa10fe4bafcb0696bde384cda5ec59188ff2"} Jul 03 10:05:25.486056 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.486026 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" event={"ID":"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6","Type":"ContainerStarted","Data":"b19f4968d89fc248df5e9bad0515bacdaa0360e9a7845fc2cd454330f9b16fb5"} Jul 03 10:05:25.487070 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:25.487047 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g55zz" event={"ID":"57e21cd6-3595-4064-95a5-2b22c0824be6","Type":"ContainerStarted","Data":"0f816df7099563fa500f66ce512a26e72799f14b79e5bd1dd1f544e0fcb11f15"} Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:26.151373 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:26.151620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkl8\" (UniqueName: \"kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8\") pod \"network-check-target-hkgr6\" (UID: \"12d45bd8-f937-4c62-98a4-63a18defb862\") " pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:26.151676 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:26.151819 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:26.151881 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs podName:ad84ff02-cefd-4c8e-b897-8a97a3727b79 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:28.151862835 +0000 UTC m=+6.217938022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs") pod "network-metrics-daemon-2rjc5" (UID: "ad84ff02-cefd-4c8e-b897-8a97a3727b79") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:26.152269 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:26.152321 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret podName:108d4f92-ca93-47d8-a45f-ba7088301bfa nodeName:}" failed. No retries permitted until 2026-07-03 10:05:28.152307622 +0000 UTC m=+6.218382806 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret") pod "global-pull-secret-syncer-vz6gj" (UID: "108d4f92-ca93-47d8-a45f-ba7088301bfa") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:26.152396 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:26.152410 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:26.152423 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cqkl8 for pod openshift-network-diagnostics/network-check-target-hkgr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:26.152526 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:26.152457 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8 podName:12d45bd8-f937-4c62-98a4-63a18defb862 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:28.152446366 +0000 UTC m=+6.218521537 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqkl8" (UniqueName: "kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8") pod "network-check-target-hkgr6" (UID: "12d45bd8-f937-4c62-98a4-63a18defb862") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:26.367463 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:26.367364 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 10:00:24 +0000 UTC" deadline="2028-02-14 11:25:17.71825219 +0000 UTC" Jul 03 10:05:26.367463 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:26.367404 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14185h19m51.350853106s" Jul 03 10:05:26.469033 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:26.468421 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:26.469033 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:26.468550 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:26.469033 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:26.468877 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:26.469033 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:26.468995 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:26.513594 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:26.513555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" event={"ID":"40608af7edd160b369e43165ab923d25","Type":"ContainerStarted","Data":"da22fdc81e4edfedea6c56ee9c74b821b2cd3c13d1f4d0c7f8c03baa94e09671"} Jul 03 10:05:27.466006 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:27.465972 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:27.466220 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:27.466126 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:27.519644 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:27.519609 2577 generic.go:356] "Generic (PLEG): container finished" podID="543c32978d2cab93351cc69b95518e92" containerID="5a10d39447d4f31a4fe2c57ecc4cd974dc507c1f688429517149dda707a8fa75" exitCode=0 Jul 03 10:05:27.520751 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:27.520667 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" event={"ID":"543c32978d2cab93351cc69b95518e92","Type":"ContainerDied","Data":"5a10d39447d4f31a4fe2c57ecc4cd974dc507c1f688429517149dda707a8fa75"} Jul 03 10:05:27.541780 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:27.541721 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" podStartSLOduration=4.541700191 podStartE2EDuration="4.541700191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:26.536948184 +0000 UTC m=+4.603023377" watchObservedRunningTime="2026-07-03 10:05:27.541700191 +0000 UTC m=+5.607775383" Jul 03 10:05:28.173974 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:28.173885 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:28.173974 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:28.173943 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkl8\" (UniqueName: \"kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8\") pod \"network-check-target-hkgr6\" (UID: \"12d45bd8-f937-4c62-98a4-63a18defb862\") " pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:28.174223 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:28.173984 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:28.174223 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:28.174148 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:28.174223 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:28.174214 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs podName:ad84ff02-cefd-4c8e-b897-8a97a3727b79 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:32.174195415 +0000 UTC m=+10.240270589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs") pod "network-metrics-daemon-2rjc5" (UID: "ad84ff02-cefd-4c8e-b897-8a97a3727b79") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:28.174629 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:28.174607 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:28.174700 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:28.174668 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret podName:108d4f92-ca93-47d8-a45f-ba7088301bfa nodeName:}" failed. No retries permitted until 2026-07-03 10:05:32.174652369 +0000 UTC m=+10.240727542 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret") pod "global-pull-secret-syncer-vz6gj" (UID: "108d4f92-ca93-47d8-a45f-ba7088301bfa") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:28.174763 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:28.174748 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:28.174811 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:28.174764 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:28.174811 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:28.174779 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cqkl8 for pod openshift-network-diagnostics/network-check-target-hkgr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:28.174903 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:28.174817 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8 podName:12d45bd8-f937-4c62-98a4-63a18defb862 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:32.174803329 +0000 UTC m=+10.240878500 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqkl8" (UniqueName: "kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8") pod "network-check-target-hkgr6" (UID: "12d45bd8-f937-4c62-98a4-63a18defb862") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:28.469957 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:28.469213 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:28.469957 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:28.469345 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:28.469957 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:28.469717 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:28.469957 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:28.469853 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:28.526754 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:28.526061 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" event={"ID":"543c32978d2cab93351cc69b95518e92","Type":"ContainerStarted","Data":"2dcf8de66032602e379447745e35cee9b98686a7043d41707ac044b1124e6224"} Jul 03 10:05:29.465923 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:29.465888 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:29.466114 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:29.466016 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:30.466217 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:30.466178 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:30.466217 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:30.466201 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:30.466742 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:30.466295 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:30.466742 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:30.466447 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:31.466107 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:31.466067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:31.466313 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:31.466219 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:32.208444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:32.208502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkl8\" (UniqueName: \"kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8\") pod \"network-check-target-hkgr6\" (UID: \"12d45bd8-f937-4c62-98a4-63a18defb862\") " pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:32.208541 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:32.208658 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:32.208716 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs podName:ad84ff02-cefd-4c8e-b897-8a97a3727b79 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:40.208697801 +0000 UTC m=+18.274772976 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs") pod "network-metrics-daemon-2rjc5" (UID: "ad84ff02-cefd-4c8e-b897-8a97a3727b79") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:32.208862 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:32.208880 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:32.208891 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cqkl8 for pod openshift-network-diagnostics/network-check-target-hkgr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:32.208932 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8 podName:12d45bd8-f937-4c62-98a4-63a18defb862 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:40.208920288 +0000 UTC m=+18.274995474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqkl8" (UniqueName: "kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8") pod "network-check-target-hkgr6" (UID: "12d45bd8-f937-4c62-98a4-63a18defb862") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:32.208986 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:32.209045 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:32.209017 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret podName:108d4f92-ca93-47d8-a45f-ba7088301bfa nodeName:}" failed. No retries permitted until 2026-07-03 10:05:40.209007307 +0000 UTC m=+18.275082493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret") pod "global-pull-secret-syncer-vz6gj" (UID: "108d4f92-ca93-47d8-a45f-ba7088301bfa") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:32.466931 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:32.466850 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:32.467386 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:32.466978 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:32.467386 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:32.467308 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:32.467386 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:32.467370 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:33.465781 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:33.465613 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:33.465954 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:33.465785 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:34.465638 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:34.465602 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:34.466138 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:34.465753 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:34.466138 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:34.465793 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:34.466138 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:34.465887 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:35.465955 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:35.465909 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:35.466431 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:35.466051 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:36.466310 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:36.466273 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:36.466310 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:36.466319 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:36.466793 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:36.466411 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:36.466793 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:36.466518 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:37.466593 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:37.466554 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:37.467069 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:37.466690 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:38.465629 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:38.465590 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:38.465827 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:38.465591 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:38.465827 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:38.465744 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:38.465827 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:38.465793 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:39.466526 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:39.466491 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:39.466954 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:39.466621 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:40.267380 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:40.267333 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:40.267546 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:40.267402 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkl8\" (UniqueName: \"kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8\") pod \"network-check-target-hkgr6\" (UID: \"12d45bd8-f937-4c62-98a4-63a18defb862\") " pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:40.267546 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:40.267445 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:40.267546 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:40.267488 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:40.267723 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:40.267556 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:40.267723 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:40.267567 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret podName:108d4f92-ca93-47d8-a45f-ba7088301bfa nodeName:}" failed. No retries permitted until 2026-07-03 10:05:56.267544559 +0000 UTC m=+34.333619748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret") pod "global-pull-secret-syncer-vz6gj" (UID: "108d4f92-ca93-47d8-a45f-ba7088301bfa") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:40.267723 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:40.267572 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:40.267723 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:40.267592 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:40.267723 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:40.267605 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cqkl8 for pod openshift-network-diagnostics/network-check-target-hkgr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:40.267723 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:40.267610 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs podName:ad84ff02-cefd-4c8e-b897-8a97a3727b79 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:56.267593012 +0000 UTC m=+34.333668182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs") pod "network-metrics-daemon-2rjc5" (UID: "ad84ff02-cefd-4c8e-b897-8a97a3727b79") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:40.267723 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:40.267640 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8 podName:12d45bd8-f937-4c62-98a4-63a18defb862 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:56.267630877 +0000 UTC m=+34.333706065 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqkl8" (UniqueName: "kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8") pod "network-check-target-hkgr6" (UID: "12d45bd8-f937-4c62-98a4-63a18defb862") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:40.466581 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:40.466495 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:40.466581 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:40.466506 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:40.467018 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:40.466646 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:40.467018 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:40.466793 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:41.465957 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:41.465924 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:41.466139 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:41.466051 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:42.466992 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:42.466957 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:42.467430 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:42.467110 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:42.467430 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:42.467158 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:42.467430 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:42.467248 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:43.465934 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:43.465892 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:43.466124 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:43.466014 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:44.465788 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.465551 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:44.466510 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.465614 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:44.466510 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:44.465817 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:44.466510 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:44.465881 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:44.559365 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.559331 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" event={"ID":"c0ae8a49-7ced-4ffa-8757-d5034a4b3752","Type":"ContainerStarted","Data":"c3abdfd7e89557607ad2b6b87c659885404a2cc5262bd65cea30abe416cc6f19"} Jul 03 10:05:44.560800 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.560771 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rrn2q" event={"ID":"ef4101c3-8dd9-450e-96d1-e7e8159cf062","Type":"ContainerStarted","Data":"5210fcaba07488550d798537805a12ec43a470abe37f5515e1e47b02407eadfc"} Jul 03 10:05:44.562044 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.562019 2577 generic.go:356] "Generic (PLEG): container finished" podID="3a89dafb-6d42-40f3-b9d4-cb4a0c839af6" containerID="ba464270a00a8742cc1c5364b31570acc5de99e1b0de8d0a8f145da39c5fbddc" exitCode=0 Jul 03 10:05:44.562169 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.562112 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" event={"ID":"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6","Type":"ContainerDied","Data":"ba464270a00a8742cc1c5364b31570acc5de99e1b0de8d0a8f145da39c5fbddc"} Jul 03 10:05:44.563668 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.563623 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g55zz" event={"ID":"57e21cd6-3595-4064-95a5-2b22c0824be6","Type":"ContainerStarted","Data":"a4bce4d64e4ca82ffdbdc023a61f0590bcc55a9d10d765832e6fc82fa46a745a"} Jul 03 10:05:44.565031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.565006 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" event={"ID":"25536018-295a-44e5-b396-44d6f533d2b2","Type":"ContainerStarted","Data":"6473714c3cc6c779d1dc2d783e197db0bfb4e437b20f804d1e0485928ee37bf2"} Jul 03 10:05:44.566500 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.566480 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bm5md" event={"ID":"01500b78-d50b-467c-bf74-f4b3e310e159","Type":"ContainerStarted","Data":"14f0567f591f88a0baf86e74ca7bdcc9656a2b791e64a066f67b61a1e6720a42"} Jul 03 10:05:44.567851 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.567824 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pq999" event={"ID":"c8bd73b8-7b17-4f0b-a23b-d343da75dd3a","Type":"ContainerStarted","Data":"3f98f057860e694f3c1598e50a8ee582f2632583701fbe1a3f86e30780b58702"} Jul 03 10:05:44.569313 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.569292 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kzm94" event={"ID":"75626273-af77-4838-99cc-2d3b274069d4","Type":"ContainerStarted","Data":"0173e242cd5b4943e1388852bb5f79c6f9adc2c5a9bf863ee784f60e740e545e"} Jul 03 10:05:44.587849 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.587809 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" podStartSLOduration=21.587794408 podStartE2EDuration="21.587794408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:28.547371944 +0000 UTC m=+6.613447137" watchObservedRunningTime="2026-07-03 10:05:44.587794408 +0000 UTC m=+22.653869599" Jul 03 10:05:44.606282 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.606237 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g55zz" podStartSLOduration=2.97496723 podStartE2EDuration="22.606221411s" totalImagesPullingTime="19.631254181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:44.605641111 +0000 UTC m=+22.671716304" watchObservedRunningTime="2026-07-03 10:05:44.606221411 +0000 UTC m=+22.672296603" Jul 03 10:05:44.626676 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.626626 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kzm94" podStartSLOduration=11.51184774 podStartE2EDuration="22.62661209s" totalImagesPullingTime="11.11476435s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:44.626452935 +0000 UTC m=+22.692528128" watchObservedRunningTime="2026-07-03 10:05:44.62661209 +0000 UTC m=+22.692687281" Jul 03 10:05:44.646691 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.646638 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rrn2q" podStartSLOduration=11.580302741 podStartE2EDuration="22.646623348s" totalImagesPullingTime="11.066320607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:44.646028851 +0000 UTC m=+22.712104045" watchObservedRunningTime="2026-07-03 10:05:44.646623348 +0000 UTC m=+22.712698539" Jul 03 10:05:44.669024 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.668976 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pq999" podStartSLOduration=3.17438276 podStartE2EDuration="22.668961111s" totalImagesPullingTime="19.494578351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:44.668394051 +0000 UTC m=+22.734469244" watchObservedRunningTime="2026-07-03 10:05:44.668961111 +0000 UTC m=+22.735036303" Jul 03 10:05:44.686886 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:44.686838 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bm5md" podStartSLOduration=11.62749647 podStartE2EDuration="22.686823075s" totalImagesPullingTime="11.059326605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:44.68625536 +0000 UTC m=+22.752330553" watchObservedRunningTime="2026-07-03 10:05:44.686823075 +0000 UTC m=+22.752898266" Jul 03 10:05:45.466333 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:45.466147 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:45.466941 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:45.466416 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:45.573875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:45.573845 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:05:45.574225 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:45.574187 2577 generic.go:356] "Generic (PLEG): container finished" podID="25536018-295a-44e5-b396-44d6f533d2b2" containerID="e1dc52aad9c0c361efb769edefd0b4254478ec264ff447214121ac01c1a0dc75" exitCode=1 Jul 03 10:05:45.574347 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:45.574275 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" event={"ID":"25536018-295a-44e5-b396-44d6f533d2b2","Type":"ContainerStarted","Data":"6ea0b9f80cb1c902335ddb466f9c7a398d15eb4fb5cfcb65f2cb01a545331164"} Jul 03 10:05:45.574398 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:45.574348 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" event={"ID":"25536018-295a-44e5-b396-44d6f533d2b2","Type":"ContainerStarted","Data":"ce6a52341fd61005c6adabb42a904ecd63b8487d5d1cda7f4fb5b7304deb92f2"} Jul 03 10:05:45.574443 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:45.574397 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" event={"ID":"25536018-295a-44e5-b396-44d6f533d2b2","Type":"ContainerStarted","Data":"da4a0163ec8d0d7051ef9abc6657a13ab544c05321e472de350758d34970f48b"} Jul 03 10:05:45.574478 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:45.574447 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" event={"ID":"25536018-295a-44e5-b396-44d6f533d2b2","Type":"ContainerStarted","Data":"bf3d34b74deef653db7d04e20628c98ab45e8c04bde574e218463cd67b4f17ed"} Jul 03 10:05:45.574519 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:45.574494 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" event={"ID":"25536018-295a-44e5-b396-44d6f533d2b2","Type":"ContainerDied","Data":"e1dc52aad9c0c361efb769edefd0b4254478ec264ff447214121ac01c1a0dc75"} Jul 03 10:05:45.959028 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:45.958993 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 03 10:05:46.407554 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:46.407458 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-03T10:05:45.959019123Z","UUID":"cb9fc7a8-565f-4a2c-8212-9b55802db4ee","Handler":null,"Name":"","Endpoint":""} Jul 03 10:05:46.408890 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:46.408871 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 03 10:05:46.409001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:46.408910 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 03 10:05:46.465552 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:46.465517 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:46.465734 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:46.465574 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:46.465734 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:46.465647 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:46.465817 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:46.465778 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:46.577911 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:46.577879 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fwkpz" event={"ID":"45b940e2-1138-4ed4-95a8-db85ef6328b2","Type":"ContainerStarted","Data":"703a77ef5438f28a84918addee762356969f51c8be5c16707862cb06340977d6"} Jul 03 10:05:46.579559 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:46.579532 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" event={"ID":"c0ae8a49-7ced-4ffa-8757-d5034a4b3752","Type":"ContainerStarted","Data":"d371f4daa059f6c81984bf67de743886fc4b35806977e687ed9c00b262c921cc"} Jul 03 10:05:46.593719 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:46.593665 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fwkpz" podStartSLOduration=13.515179355 podStartE2EDuration="24.593650332s" totalImagesPullingTime="11.078470977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:46.593241548 +0000 UTC m=+24.659316741" watchObservedRunningTime="2026-07-03 10:05:46.593650332 +0000 UTC m=+24.659725523" Jul 03 10:05:47.466714 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:47.466458 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:47.466906 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:47.466835 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:47.586382 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:47.586356 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:05:47.586857 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:47.586830 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" event={"ID":"25536018-295a-44e5-b396-44d6f533d2b2","Type":"ContainerStarted","Data":"8bf7ae17a8d1c996aa3e258eb71623fd1c7d3b91cc357487a99163bd03686861"} Jul 03 10:05:47.589217 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:47.589188 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" event={"ID":"c0ae8a49-7ced-4ffa-8757-d5034a4b3752","Type":"ContainerStarted","Data":"f32b746671fa771ca0cb74b747efcc491f534830692c296f811587352e252460"} Jul 03 10:05:47.611543 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:47.611487 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9hnh" podStartSLOduration=11.633978484 podStartE2EDuration="25.611470945s" totalImagesPullingTime="13.977492461s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:47.611356824 +0000 UTC m=+25.677432021" watchObservedRunningTime="2026-07-03 10:05:47.611470945 +0000 UTC m=+25.677546136" Jul 03 10:05:48.465693 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:48.465661 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:48.465693 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:48.465681 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:48.465933 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:48.465801 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:48.465988 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:48.465935 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:49.349832 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:49.349795 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:49.350577 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:49.350456 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:49.465581 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:49.465546 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:49.465753 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:49.465693 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:49.594679 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:49.594650 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:49.595174 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:49.595154 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rrn2q" Jul 03 10:05:50.468553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:50.468531 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:50.468955 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:50.468640 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:50.468955 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:50.468716 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:50.468955 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:50.468802 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:50.601451 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:50.601056 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:05:50.601951 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:50.601692 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" event={"ID":"25536018-295a-44e5-b396-44d6f533d2b2","Type":"ContainerStarted","Data":"e32da85087846f85ade8f45bc43d7e8666e32d05478f9905b9eedbf0a9853493"} Jul 03 10:05:50.602420 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:50.602213 2577 scope.go:117] "RemoveContainer" containerID="e1dc52aad9c0c361efb769edefd0b4254478ec264ff447214121ac01c1a0dc75" Jul 03 10:05:51.465531 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.465504 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:51.465631 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:51.465611 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:51.606110 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.606070 2577 generic.go:356] "Generic (PLEG): container finished" podID="3a89dafb-6d42-40f3-b9d4-cb4a0c839af6" containerID="4b25bc7e3a75e21f62ecf64ce79c52aa5300336668e1ec86e08c5917f140836e" exitCode=0 Jul 03 10:05:51.606549 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.606127 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" event={"ID":"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6","Type":"ContainerDied","Data":"4b25bc7e3a75e21f62ecf64ce79c52aa5300336668e1ec86e08c5917f140836e"} Jul 03 10:05:51.609809 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.609791 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:05:51.610200 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.610176 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" event={"ID":"25536018-295a-44e5-b396-44d6f533d2b2","Type":"ContainerStarted","Data":"239f07c8589629de2293694e319bd9175a9b2e84b606998d0952a6f98a64eebe"} Jul 03 10:05:51.610453 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.610406 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:51.610619 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.610604 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:51.610662 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.610653 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:51.626213 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.626187 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:51.626382 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.626321 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:05:51.668313 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:51.668263 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" podStartSLOduration=9.974651857 podStartE2EDuration="29.668247572s" totalImagesPullingTime="19.693595715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:51.666436063 +0000 UTC m=+29.732511256" watchObservedRunningTime="2026-07-03 10:05:51.668247572 +0000 UTC m=+29.734322763" Jul 03 10:05:52.466920 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:52.466891 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:52.467128 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:52.466988 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:52.467128 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:52.467000 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:52.467128 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:52.467088 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:52.527038 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:52.527004 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hkgr6"] Jul 03 10:05:52.527831 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:52.527669 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vz6gj"] Jul 03 10:05:52.527831 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:52.527778 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:52.528002 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:52.527863 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:52.530425 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:52.530404 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2rjc5"] Jul 03 10:05:52.615158 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:52.614891 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:52.615539 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:52.614937 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" event={"ID":"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6","Type":"ContainerStarted","Data":"c44743ec794a9ef6c39f71e888979dfaf871908975bfe6e8a93b15e04358a548"} Jul 03 10:05:52.615539 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:52.615295 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:52.615808 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:52.615789 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:52.615890 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:52.615873 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:53.619384 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:53.619347 2577 generic.go:356] "Generic (PLEG): container finished" podID="3a89dafb-6d42-40f3-b9d4-cb4a0c839af6" containerID="c44743ec794a9ef6c39f71e888979dfaf871908975bfe6e8a93b15e04358a548" exitCode=0 Jul 03 10:05:53.619384 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:53.619377 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" event={"ID":"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6","Type":"ContainerDied","Data":"c44743ec794a9ef6c39f71e888979dfaf871908975bfe6e8a93b15e04358a548"} Jul 03 10:05:54.465991 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:54.465914 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:54.465991 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:54.465939 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:54.466194 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:54.466020 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:54.466194 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:54.466031 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:54.466194 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:54.466089 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:54.466194 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:54.466168 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:55.628912 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:55.628878 2577 generic.go:356] "Generic (PLEG): container finished" podID="3a89dafb-6d42-40f3-b9d4-cb4a0c839af6" containerID="925457abbd737195c576bcecfce3c2a69c5c1ef8a9daff6410afdea540a06ecb" exitCode=0 Jul 03 10:05:55.629536 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:55.628918 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" event={"ID":"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6","Type":"ContainerDied","Data":"925457abbd737195c576bcecfce3c2a69c5c1ef8a9daff6410afdea540a06ecb"} Jul 03 10:05:56.290264 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:56.290231 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:56.290264 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:56.290266 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkl8\" (UniqueName: \"kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8\") pod \"network-check-target-hkgr6\" (UID: \"12d45bd8-f937-4c62-98a4-63a18defb862\") " pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:56.290478 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:56.290291 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:56.290478 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.290386 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:56.290478 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.290386 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:56.290478 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.290398 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:56.290478 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.290420 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:56.290478 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.290432 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cqkl8 for pod openshift-network-diagnostics/network-check-target-hkgr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:56.290478 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.290432 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs podName:ad84ff02-cefd-4c8e-b897-8a97a3727b79 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:28.2904197 +0000 UTC m=+66.356494870 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs") pod "network-metrics-daemon-2rjc5" (UID: "ad84ff02-cefd-4c8e-b897-8a97a3727b79") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:56.290478 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.290474 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret podName:108d4f92-ca93-47d8-a45f-ba7088301bfa nodeName:}" failed. No retries permitted until 2026-07-03 10:06:28.290461399 +0000 UTC m=+66.356536572 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret") pod "global-pull-secret-syncer-vz6gj" (UID: "108d4f92-ca93-47d8-a45f-ba7088301bfa") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:56.290478 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.290486 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8 podName:12d45bd8-f937-4c62-98a4-63a18defb862 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:28.290479584 +0000 UTC m=+66.356554757 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqkl8" (UniqueName: "kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8") pod "network-check-target-hkgr6" (UID: "12d45bd8-f937-4c62-98a4-63a18defb862") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:56.466491 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:56.466458 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:56.466668 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:56.466464 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:56.466668 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.466567 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2rjc5" podUID="ad84ff02-cefd-4c8e-b897-8a97a3727b79" Jul 03 10:05:56.466668 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.466656 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6gj" podUID="108d4f92-ca93-47d8-a45f-ba7088301bfa" Jul 03 10:05:56.466792 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:56.466467 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:56.466792 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:56.466749 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkgr6" podUID="12d45bd8-f937-4c62-98a4-63a18defb862" Jul 03 10:05:58.215276 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.215251 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeReady" Jul 03 10:05:58.215712 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.215375 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 03 10:05:58.250600 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.250564 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5847694c56-wkv9n"] Jul 03 10:05:58.252834 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.252813 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.260453 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.257705 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 03 10:05:58.260453 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.257741 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zmbwq\"" Jul 03 10:05:58.260453 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.259086 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 03 10:05:58.260453 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.259086 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 03 10:05:58.260453 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.259599 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-tbrbs"] Jul 03 10:05:58.264585 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.264559 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988"] Jul 03 10:05:58.264724 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.264681 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.266937 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.266915 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 03 10:05:58.267037 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.266940 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5"] Jul 03 10:05:58.267214 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.267067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:05:58.268976 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.268954 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 03 10:05:58.269082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.268997 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 03 10:05:58.269082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.269018 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kz8kn\"" Jul 03 10:05:58.269082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.268960 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 03 10:05:58.269528 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.269507 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr"] Jul 03 10:05:58.269619 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.269601 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:05:58.269732 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.269626 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 03 10:05:58.271599 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.271580 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7f7ff5477c-6gn75"] Jul 03 10:05:58.271774 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.271757 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr" Jul 03 10:05:58.274974 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.274376 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.275532 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.275513 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 03 10:05:58.275932 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.275516 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bdd9n"] Jul 03 10:05:58.275932 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.275691 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 03 10:05:58.275932 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.275862 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:05:58.278236 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.278176 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7xc64"] Jul 03 10:05:58.278733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.278714 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 03 10:05:58.278976 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.278956 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 03 10:05:58.279577 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.279558 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r2dls\"" Jul 03 10:05:58.279659 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.279620 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 03 10:05:58.279786 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.279772 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 03 10:05:58.279879 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.279863 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 03 10:05:58.280003 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.279980 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6"] Jul 03 10:05:58.280133 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.280068 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-g8qr5\"" Jul 03 10:05:58.280376 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.280284 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.284688 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.280828 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7xc64" Jul 03 10:05:58.284688 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.281144 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 03 10:05:58.284688 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.281205 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:05:58.284688 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.281817 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-k5pgd\"" Jul 03 10:05:58.284688 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.282846 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 03 10:05:58.284688 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.283289 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 03 10:05:58.285259 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.284889 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g"] Jul 03 10:05:58.285835 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.285814 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.286899 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.286880 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zg8hw\"" Jul 03 10:05:58.287392 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.287376 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 03 10:05:58.288331 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.288311 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 03 10:05:58.288477 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.288458 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.288583 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.288547 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 03 10:05:58.289540 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.289520 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 03 10:05:58.290291 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.290255 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 03 10:05:58.290453 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.290403 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 03 10:05:58.292472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.290998 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 03 10:05:58.292472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.291027 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sl8pb\"" Jul 03 10:05:58.292472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.291410 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 03 10:05:58.292472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.291420 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 03 10:05:58.292472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.288317 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-8mlv5"] Jul 03 10:05:58.292472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.291796 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 03 10:05:58.292472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.291937 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sgbfl\"" Jul 03 10:05:58.292472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.291969 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:05:58.293211 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.293158 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 03 10:05:58.293460 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.293443 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 03 10:05:58.295258 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.294874 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:05:58.295258 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.295061 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9r5cl\"" Jul 03 10:05:58.295427 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.295309 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-fcfpb\"" Jul 03 10:05:58.297503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.297480 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 03 10:05:58.297751 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.297736 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 03 10:05:58.297855 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.297742 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 03 10:05:58.299654 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.299629 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-sct46"] Jul 03 10:05:58.300392 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.300374 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.301975 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.301835 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988"] Jul 03 10:05:58.301975 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.301877 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-tbrbs"] Jul 03 10:05:58.301975 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.301901 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr"] Jul 03 10:05:58.301975 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.301923 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5847694c56-wkv9n"] Jul 03 10:05:58.301975 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.301971 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5"] Jul 03 10:05:58.302321 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.301995 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f7ff5477c-6gn75"] Jul 03 10:05:58.302321 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.302052 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-8mlv5"] Jul 03 10:05:58.302321 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.302089 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6"] Jul 03 10:05:58.302321 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.302159 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-sct46" Jul 03 10:05:58.304226 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.303344 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:05:58.304226 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.303799 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 03 10:05:58.304226 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.304048 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 03 10:05:58.304414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.304274 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wqccv\"" Jul 03 10:05:58.304414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.304400 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 03 10:05:58.304737 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.304717 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g"] Jul 03 10:05:58.304829 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.304760 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7xc64"] Jul 03 10:05:58.304940 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.304924 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bdd9n"] Jul 03 10:05:58.305963 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.305935 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 10:05:58.306059 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.306031 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 10:05:58.307164 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.307146 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vx79g\"" Jul 03 10:05:58.309155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.309133 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-sct46"] Jul 03 10:05:58.311691 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.311673 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 03 10:05:58.406862 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.406820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.406862 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.406878 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-trusted-ca\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.407146 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.406903 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwlk5\" (UniqueName: \"kubernetes.io/projected/35d12807-d620-406c-ae20-d8a5ee68e92d-kube-api-access-nwlk5\") pod \"volume-data-source-validator-5d456fdbcd-lqvkr\" (UID: \"35d12807-d620-406c-ae20-d8a5ee68e92d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr" Jul 03 10:05:58.407146 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.406956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjtqb\" (UniqueName: \"kubernetes.io/projected/3114663a-1656-4dc5-9f4d-288871787eff-kube-api-access-sjtqb\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:05:58.407146 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.406983 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmr84\" (UniqueName: \"kubernetes.io/projected/6eac509e-8324-4a69-ae4d-7923496d16a0-kube-api-access-lmr84\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:05:58.407146 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.407146 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407028 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-stats-auth\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.407468 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407429 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-certificates\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.407606 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407487 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s8x6\" (UniqueName: \"kubernetes.io/projected/41cb1751-d956-4d34-9e67-e3d994e6bf5f-kube-api-access-2s8x6\") pod \"service-ca-operator-5d46d9ffd4-gqf2g\" (UID: \"41cb1751-d956-4d34-9e67-e3d994e6bf5f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.407606 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407566 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-bound-sa-token\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.407606 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407597 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c24a47b-93f2-4783-9f16-e2fbf02d7ceb-cert\") pod \"ingress-canary-7xc64\" (UID: \"3c24a47b-93f2-4783-9f16-e2fbf02d7ceb\") " pod="openshift-ingress-canary/ingress-canary-7xc64" Jul 03 10:05:58.407762 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-snapshots\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.407762 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407649 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-serving-cert\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.407762 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407674 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p55d6\" (UniqueName: \"kubernetes.io/projected/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-kube-api-access-p55d6\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.407762 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407701 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efd9c9dc-f958-4414-bd48-3d2e6051d75c-tmp-dir\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.407762 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7e396d0-1598-4e1b-8203-660c98a64ea7-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-rhcp6\" (UID: \"a7e396d0-1598-4e1b-8203-660c98a64ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.407762 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407758 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8cc555a9-6421-474e-bcb7-399b574dc972-trusted-ca\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.408025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407779 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjw67\" (UniqueName: \"kubernetes.io/projected/efd9c9dc-f958-4414-bd48-3d2e6051d75c-kube-api-access-qjw67\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.408025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407801 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rszdh\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-kube-api-access-rszdh\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.408025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407849 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8cc555a9-6421-474e-bcb7-399b574dc972-config\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.408025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407875 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8cc555a9-6421-474e-bcb7-399b574dc972-serving-cert\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.408025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:05:58.408025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407968 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xbtp\" (UniqueName: \"kubernetes.io/projected/7a741adb-6089-40c5-ba07-7f334eadd790-kube-api-access-2xbtp\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.408025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.407989 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.408025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408003 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-installation-pull-secrets\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.408025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408018 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-tmp\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.408475 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408053 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41cb1751-d956-4d34-9e67-e3d994e6bf5f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-gqf2g\" (UID: \"41cb1751-d956-4d34-9e67-e3d994e6bf5f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.408475 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408078 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-default-certificate\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.408575 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408164 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x98f\" (UniqueName: \"kubernetes.io/projected/a7e396d0-1598-4e1b-8203-660c98a64ea7-kube-api-access-9x98f\") pod \"kube-storage-version-migrator-operator-846bf668f7-rhcp6\" (UID: \"a7e396d0-1598-4e1b-8203-660c98a64ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.408632 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408572 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.408632 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408590 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nbfs\" (UniqueName: \"kubernetes.io/projected/7ef7157e-cc50-408f-a010-9d6d78d6ca5e-kube-api-access-5nbfs\") pod \"network-check-source-989f8b8dc-sct46\" (UID: \"7ef7157e-cc50-408f-a010-9d6d78d6ca5e\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-sct46" Jul 03 10:05:58.408632 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408610 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-image-registry-private-configuration\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.408632 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408624 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efd9c9dc-f958-4414-bd48-3d2e6051d75c-config-volume\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.408797 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408642 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-ca-trust-extracted\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.408797 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408657 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw4ks\" (UniqueName: \"kubernetes.io/projected/8cc555a9-6421-474e-bcb7-399b574dc972-kube-api-access-dw4ks\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.408797 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408679 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6eac509e-8324-4a69-ae4d-7923496d16a0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:05:58.408797 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.408797 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408708 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.408797 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408723 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41cb1751-d956-4d34-9e67-e3d994e6bf5f-config\") pod \"service-ca-operator-5d46d9ffd4-gqf2g\" (UID: \"41cb1751-d956-4d34-9e67-e3d994e6bf5f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.408797 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408759 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7e396d0-1598-4e1b-8203-660c98a64ea7-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-rhcp6\" (UID: \"a7e396d0-1598-4e1b-8203-660c98a64ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.409120 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408808 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:05:58.409120 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.408833 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs5h8\" (UniqueName: \"kubernetes.io/projected/3c24a47b-93f2-4783-9f16-e2fbf02d7ceb-kube-api-access-gs5h8\") pod \"ingress-canary-7xc64\" (UID: \"3c24a47b-93f2-4783-9f16-e2fbf02d7ceb\") " pod="openshift-ingress-canary/ingress-canary-7xc64" Jul 03 10:05:58.466368 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.466287 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:05:58.466535 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.466288 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:05:58.466535 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.466426 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:05:58.469375 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.469347 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 10:05:58.469375 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.469363 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 10:05:58.469554 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.469376 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xh5d7\"" Jul 03 10:05:58.469554 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.469377 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-njhrs\"" Jul 03 10:05:58.509247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509211 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x98f\" (UniqueName: \"kubernetes.io/projected/a7e396d0-1598-4e1b-8203-660c98a64ea7-kube-api-access-9x98f\") pod \"kube-storage-version-migrator-operator-846bf668f7-rhcp6\" (UID: \"a7e396d0-1598-4e1b-8203-660c98a64ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.509247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509252 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.509503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509277 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nbfs\" (UniqueName: \"kubernetes.io/projected/7ef7157e-cc50-408f-a010-9d6d78d6ca5e-kube-api-access-5nbfs\") pod \"network-check-source-989f8b8dc-sct46\" (UID: \"7ef7157e-cc50-408f-a010-9d6d78d6ca5e\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-sct46" Jul 03 10:05:58.509503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509305 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-image-registry-private-configuration\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.509503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509383 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efd9c9dc-f958-4414-bd48-3d2e6051d75c-config-volume\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.509503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-ca-trust-extracted\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.509503 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.509396 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 03 10:05:58.509747 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509458 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw4ks\" (UniqueName: \"kubernetes.io/projected/8cc555a9-6421-474e-bcb7-399b574dc972-kube-api-access-dw4ks\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.509747 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.509540 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:59.009514586 +0000 UTC m=+37.075589770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : secret "router-metrics-certs-default" not found Jul 03 10:05:58.509747 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6eac509e-8324-4a69-ae4d-7923496d16a0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:05:58.509747 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509625 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.509747 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.509747 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509679 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41cb1751-d956-4d34-9e67-e3d994e6bf5f-config\") pod \"service-ca-operator-5d46d9ffd4-gqf2g\" (UID: \"41cb1751-d956-4d34-9e67-e3d994e6bf5f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.509747 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509711 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7e396d0-1598-4e1b-8203-660c98a64ea7-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-rhcp6\" (UID: \"a7e396d0-1598-4e1b-8203-660c98a64ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.509747 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.509726 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 10:05:58.509747 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.509740 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847694c56-wkv9n: secret "image-registry-tls" not found Jul 03 10:05:58.509747 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509744 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509770 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs5h8\" (UniqueName: \"kubernetes.io/projected/3c24a47b-93f2-4783-9f16-e2fbf02d7ceb-kube-api-access-gs5h8\") pod \"ingress-canary-7xc64\" (UID: \"3c24a47b-93f2-4783-9f16-e2fbf02d7ceb\") " pod="openshift-ingress-canary/ingress-canary-7xc64" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.509797 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls podName:9faa5c7c-106a-49eb-9b2d-074de55a3bcd nodeName:}" failed. No retries permitted until 2026-07-03 10:05:59.009781038 +0000 UTC m=+37.075856212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls") pod "image-registry-5847694c56-wkv9n" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd") : secret "image-registry-tls" not found Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509818 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-trusted-ca\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509878 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwlk5\" (UniqueName: \"kubernetes.io/projected/35d12807-d620-406c-ae20-d8a5ee68e92d-kube-api-access-nwlk5\") pod \"volume-data-source-validator-5d456fdbcd-lqvkr\" (UID: \"35d12807-d620-406c-ae20-d8a5ee68e92d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509909 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjtqb\" (UniqueName: \"kubernetes.io/projected/3114663a-1656-4dc5-9f4d-288871787eff-kube-api-access-sjtqb\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509934 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmr84\" (UniqueName: \"kubernetes.io/projected/6eac509e-8324-4a69-ae4d-7923496d16a0-kube-api-access-lmr84\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509953 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-ca-trust-extracted\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.509960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-stats-auth\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.510037 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:59.010024018 +0000 UTC m=+37.076099205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : configmap references non-existent config key: service-ca.crt Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510074 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-certificates\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510121 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s8x6\" (UniqueName: \"kubernetes.io/projected/41cb1751-d956-4d34-9e67-e3d994e6bf5f-kube-api-access-2s8x6\") pod \"service-ca-operator-5d46d9ffd4-gqf2g\" (UID: \"41cb1751-d956-4d34-9e67-e3d994e6bf5f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.510257 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510133 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efd9c9dc-f958-4414-bd48-3d2e6051d75c-config-volume\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510177 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-bound-sa-token\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510201 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c24a47b-93f2-4783-9f16-e2fbf02d7ceb-cert\") pod \"ingress-canary-7xc64\" (UID: \"3c24a47b-93f2-4783-9f16-e2fbf02d7ceb\") " pod="openshift-ingress-canary/ingress-canary-7xc64" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510225 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-snapshots\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510252 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-serving-cert\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510281 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p55d6\" (UniqueName: \"kubernetes.io/projected/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-kube-api-access-p55d6\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510305 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efd9c9dc-f958-4414-bd48-3d2e6051d75c-tmp-dir\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510308 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41cb1751-d956-4d34-9e67-e3d994e6bf5f-config\") pod \"service-ca-operator-5d46d9ffd4-gqf2g\" (UID: \"41cb1751-d956-4d34-9e67-e3d994e6bf5f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510327 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7e396d0-1598-4e1b-8203-660c98a64ea7-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-rhcp6\" (UID: \"a7e396d0-1598-4e1b-8203-660c98a64ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510360 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8cc555a9-6421-474e-bcb7-399b574dc972-trusted-ca\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510384 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjw67\" (UniqueName: \"kubernetes.io/projected/efd9c9dc-f958-4414-bd48-3d2e6051d75c-kube-api-access-qjw67\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510405 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rszdh\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-kube-api-access-rszdh\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510432 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8cc555a9-6421-474e-bcb7-399b574dc972-config\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510440 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6eac509e-8324-4a69-ae4d-7923496d16a0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510453 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8cc555a9-6421-474e-bcb7-399b574dc972-serving-cert\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510504 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:05:58.510930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510538 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xbtp\" (UniqueName: \"kubernetes.io/projected/7a741adb-6089-40c5-ba07-7f334eadd790-kube-api-access-2xbtp\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510564 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.510588 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510594 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-installation-pull-secrets\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510637 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-tmp\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.510649 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls podName:6eac509e-8324-4a69-ae4d-7923496d16a0 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:59.010632237 +0000 UTC m=+37.076707424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c4dj5" (UID: "6eac509e-8324-4a69-ae4d-7923496d16a0") : secret "cluster-monitoring-operator-tls" not found Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41cb1751-d956-4d34-9e67-e3d994e6bf5f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-gqf2g\" (UID: \"41cb1751-d956-4d34-9e67-e3d994e6bf5f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.510721 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-default-certificate\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.511146 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.511237 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.511484 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 10:05:58.511715 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.511534 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls podName:3114663a-1656-4dc5-9f4d-288871787eff nodeName:}" failed. No retries permitted until 2026-07-03 10:05:59.011521203 +0000 UTC m=+37.077596378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8d988" (UID: "3114663a-1656-4dc5-9f4d-288871787eff") : secret "samples-operator-tls" not found Jul 03 10:05:58.512275 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.511950 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-snapshots\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.512275 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.511993 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-trusted-ca\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.512647 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.512526 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7e396d0-1598-4e1b-8203-660c98a64ea7-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-rhcp6\" (UID: \"a7e396d0-1598-4e1b-8203-660c98a64ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.512764 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.512741 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-tmp\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.512833 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.512774 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-certificates\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.512928 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.512911 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 10:05:58.513061 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:58.513047 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls podName:efd9c9dc-f958-4414-bd48-3d2e6051d75c nodeName:}" failed. No retries permitted until 2026-07-03 10:05:59.01303032 +0000 UTC m=+37.079105540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls") pod "dns-default-bdd9n" (UID: "efd9c9dc-f958-4414-bd48-3d2e6051d75c") : secret "dns-default-metrics-tls" not found Jul 03 10:05:58.514855 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.514824 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8cc555a9-6421-474e-bcb7-399b574dc972-config\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.514956 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.514866 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8cc555a9-6421-474e-bcb7-399b574dc972-trusted-ca\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.515196 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.515172 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7e396d0-1598-4e1b-8203-660c98a64ea7-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-rhcp6\" (UID: \"a7e396d0-1598-4e1b-8203-660c98a64ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.515357 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.515201 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8cc555a9-6421-474e-bcb7-399b574dc972-serving-cert\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.515455 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.515293 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-image-registry-private-configuration\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.515737 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.515698 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efd9c9dc-f958-4414-bd48-3d2e6051d75c-tmp-dir\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.516210 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.516146 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-default-certificate\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.517400 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.517376 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41cb1751-d956-4d34-9e67-e3d994e6bf5f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-gqf2g\" (UID: \"41cb1751-d956-4d34-9e67-e3d994e6bf5f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.517522 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.517408 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-serving-cert\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.517522 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.517421 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-installation-pull-secrets\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.518265 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.518241 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c24a47b-93f2-4783-9f16-e2fbf02d7ceb-cert\") pod \"ingress-canary-7xc64\" (UID: \"3c24a47b-93f2-4783-9f16-e2fbf02d7ceb\") " pod="openshift-ingress-canary/ingress-canary-7xc64" Jul 03 10:05:58.519768 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.519725 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x98f\" (UniqueName: \"kubernetes.io/projected/a7e396d0-1598-4e1b-8203-660c98a64ea7-kube-api-access-9x98f\") pod \"kube-storage-version-migrator-operator-846bf668f7-rhcp6\" (UID: \"a7e396d0-1598-4e1b-8203-660c98a64ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.520891 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.520318 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nbfs\" (UniqueName: \"kubernetes.io/projected/7ef7157e-cc50-408f-a010-9d6d78d6ca5e-kube-api-access-5nbfs\") pod \"network-check-source-989f8b8dc-sct46\" (UID: \"7ef7157e-cc50-408f-a010-9d6d78d6ca5e\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-sct46" Jul 03 10:05:58.520891 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.520847 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs5h8\" (UniqueName: \"kubernetes.io/projected/3c24a47b-93f2-4783-9f16-e2fbf02d7ceb-kube-api-access-gs5h8\") pod \"ingress-canary-7xc64\" (UID: \"3c24a47b-93f2-4783-9f16-e2fbf02d7ceb\") " pod="openshift-ingress-canary/ingress-canary-7xc64" Jul 03 10:05:58.521557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.521154 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw4ks\" (UniqueName: \"kubernetes.io/projected/8cc555a9-6421-474e-bcb7-399b574dc972-kube-api-access-dw4ks\") pod \"console-operator-68b757fb67-8mlv5\" (UID: \"8cc555a9-6421-474e-bcb7-399b574dc972\") " pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.522771 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.522746 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjtqb\" (UniqueName: \"kubernetes.io/projected/3114663a-1656-4dc5-9f4d-288871787eff-kube-api-access-sjtqb\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:05:58.523234 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.523127 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s8x6\" (UniqueName: \"kubernetes.io/projected/41cb1751-d956-4d34-9e67-e3d994e6bf5f-kube-api-access-2s8x6\") pod \"service-ca-operator-5d46d9ffd4-gqf2g\" (UID: \"41cb1751-d956-4d34-9e67-e3d994e6bf5f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.523752 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.523709 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjw67\" (UniqueName: \"kubernetes.io/projected/efd9c9dc-f958-4414-bd48-3d2e6051d75c-kube-api-access-qjw67\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:58.524193 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.524083 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-bound-sa-token\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.524557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.524516 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xbtp\" (UniqueName: \"kubernetes.io/projected/7a741adb-6089-40c5-ba07-7f334eadd790-kube-api-access-2xbtp\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.524917 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.524880 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-stats-auth\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:58.525240 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.525216 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rszdh\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-kube-api-access-rszdh\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:58.525408 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.525383 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmr84\" (UniqueName: \"kubernetes.io/projected/6eac509e-8324-4a69-ae4d-7923496d16a0-kube-api-access-lmr84\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:05:58.526974 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.526953 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p55d6\" (UniqueName: \"kubernetes.io/projected/0293a9da-1f5a-4a5a-8d5d-8619a7735b2f-kube-api-access-p55d6\") pod \"insights-operator-5c847dcfc4-tbrbs\" (UID: \"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f\") " pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.527174 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.527158 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwlk5\" (UniqueName: \"kubernetes.io/projected/35d12807-d620-406c-ae20-d8a5ee68e92d-kube-api-access-nwlk5\") pod \"volume-data-source-validator-5d456fdbcd-lqvkr\" (UID: \"35d12807-d620-406c-ae20-d8a5ee68e92d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr" Jul 03 10:05:58.580294 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.580258 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" Jul 03 10:05:58.613140 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.612932 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr" Jul 03 10:05:58.636023 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.635994 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7xc64" Jul 03 10:05:58.644320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.644149 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" Jul 03 10:05:58.650232 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.650167 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" Jul 03 10:05:58.657948 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.657896 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:05:58.664922 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.664710 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-sct46" Jul 03 10:05:58.814716 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.814644 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-tbrbs"] Jul 03 10:05:58.824018 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:58.823964 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0293a9da_1f5a_4a5a_8d5d_8619a7735b2f.slice/crio-f814d1010a1a46d0da250bff13522257cbc112f8f38351edcc7ae8acbd2fe19c WatchSource:0}: Error finding container f814d1010a1a46d0da250bff13522257cbc112f8f38351edcc7ae8acbd2fe19c: Status 404 returned error can't find the container with id f814d1010a1a46d0da250bff13522257cbc112f8f38351edcc7ae8acbd2fe19c Jul 03 10:05:58.859071 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.859005 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr"] Jul 03 10:05:58.875927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.875858 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7xc64"] Jul 03 10:05:58.912480 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.912450 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6"] Jul 03 10:05:58.917166 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:58.917136 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e396d0_1598_4e1b_8203_660c98a64ea7.slice/crio-7c56f67fe893a9e5560527927e6d31a64ec83ff828b9e057ad0ed17898ae82a7 WatchSource:0}: Error finding container 7c56f67fe893a9e5560527927e6d31a64ec83ff828b9e057ad0ed17898ae82a7: Status 404 returned error can't find the container with id 7c56f67fe893a9e5560527927e6d31a64ec83ff828b9e057ad0ed17898ae82a7 Jul 03 10:05:58.927489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:58.927233 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g"] Jul 03 10:05:58.929229 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:58.929205 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41cb1751_d956_4d34_9e67_e3d994e6bf5f.slice/crio-75aaa756986f637297e26386e3732c1a370a04d6ae0d4bc69f5651680ce9124f WatchSource:0}: Error finding container 75aaa756986f637297e26386e3732c1a370a04d6ae0d4bc69f5651680ce9124f: Status 404 returned error can't find the container with id 75aaa756986f637297e26386e3732c1a370a04d6ae0d4bc69f5651680ce9124f Jul 03 10:05:59.016331 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.016236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:05:59.016331 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.016288 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:05:59.016555 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016417 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 10:05:59.016555 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.016421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:59.016555 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016492 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls podName:3114663a-1656-4dc5-9f4d-288871787eff nodeName:}" failed. No retries permitted until 2026-07-03 10:06:00.016472518 +0000 UTC m=+38.082547720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8d988" (UID: "3114663a-1656-4dc5-9f4d-288871787eff") : secret "samples-operator-tls" not found Jul 03 10:05:59.016555 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016498 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 03 10:05:59.016555 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016551 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:00.016536109 +0000 UTC m=+38.082611280 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : secret "router-metrics-certs-default" not found Jul 03 10:05:59.016759 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016551 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 10:05:59.016759 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.016576 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:05:59.016759 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016621 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls podName:efd9c9dc-f958-4414-bd48-3d2e6051d75c nodeName:}" failed. No retries permitted until 2026-07-03 10:06:00.016606213 +0000 UTC m=+38.082681395 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls") pod "dns-default-bdd9n" (UID: "efd9c9dc-f958-4414-bd48-3d2e6051d75c") : secret "dns-default-metrics-tls" not found Jul 03 10:05:59.016759 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.016648 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:05:59.016759 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016665 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 10:05:59.016759 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016676 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847694c56-wkv9n: secret "image-registry-tls" not found Jul 03 10:05:59.016759 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.016699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:05:59.016759 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016710 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls podName:9faa5c7c-106a-49eb-9b2d-074de55a3bcd nodeName:}" failed. No retries permitted until 2026-07-03 10:06:00.016698734 +0000 UTC m=+38.082773905 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls") pod "image-registry-5847694c56-wkv9n" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd") : secret "image-registry-tls" not found Jul 03 10:05:59.016759 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016717 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 10:05:59.016759 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016755 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls podName:6eac509e-8324-4a69-ae4d-7923496d16a0 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:00.016746865 +0000 UTC m=+38.082822050 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c4dj5" (UID: "6eac509e-8324-4a69-ae4d-7923496d16a0") : secret "cluster-monitoring-operator-tls" not found Jul 03 10:05:59.017134 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:05:59.016772 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:00.016764343 +0000 UTC m=+38.082839513 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : configmap references non-existent config key: service-ca.crt Jul 03 10:05:59.147445 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.147388 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-sct46"] Jul 03 10:05:59.149947 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:05:59.149886 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef7157e_cc50_408f_a010_9d6d78d6ca5e.slice/crio-f338cf728608739173add741a5d6e136a3466eb99f6b5bbc8b13965349009487 WatchSource:0}: Error finding container f338cf728608739173add741a5d6e136a3466eb99f6b5bbc8b13965349009487: Status 404 returned error can't find the container with id f338cf728608739173add741a5d6e136a3466eb99f6b5bbc8b13965349009487 Jul 03 10:05:59.151324 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.151307 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-8mlv5"] Jul 03 10:05:59.641951 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.641909 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" event={"ID":"41cb1751-d956-4d34-9e67-e3d994e6bf5f","Type":"ContainerStarted","Data":"75aaa756986f637297e26386e3732c1a370a04d6ae0d4bc69f5651680ce9124f"} Jul 03 10:05:59.643284 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.643223 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr" event={"ID":"35d12807-d620-406c-ae20-d8a5ee68e92d","Type":"ContainerStarted","Data":"5e3d08082f750be13ffc68d8cc3368664e1501cedb61193c493336aff631e38a"} Jul 03 10:05:59.644575 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.644541 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" event={"ID":"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f","Type":"ContainerStarted","Data":"f814d1010a1a46d0da250bff13522257cbc112f8f38351edcc7ae8acbd2fe19c"} Jul 03 10:05:59.645850 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.645815 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-sct46" event={"ID":"7ef7157e-cc50-408f-a010-9d6d78d6ca5e","Type":"ContainerStarted","Data":"f338cf728608739173add741a5d6e136a3466eb99f6b5bbc8b13965349009487"} Jul 03 10:05:59.647150 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.647119 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" event={"ID":"a7e396d0-1598-4e1b-8203-660c98a64ea7","Type":"ContainerStarted","Data":"7c56f67fe893a9e5560527927e6d31a64ec83ff828b9e057ad0ed17898ae82a7"} Jul 03 10:05:59.648470 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:05:59.648444 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7xc64" event={"ID":"3c24a47b-93f2-4783-9f16-e2fbf02d7ceb","Type":"ContainerStarted","Data":"df9bb369c383bd9ea1251d08873eab75254e32d1bb4a6d4b16f2a6adbc562ada"} Jul 03 10:06:00.026644 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:00.026603 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:00.026844 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:00.026701 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:06:00.026844 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:00.026733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:06:00.026844 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.026785 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:02.026766629 +0000 UTC m=+40.092841818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : configmap references non-existent config key: service-ca.crt Jul 03 10:06:00.026844 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:00.026821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:00.026844 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.026839 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 10:06:00.027112 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:00.026864 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:06:00.027112 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.026876 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls podName:efd9c9dc-f958-4414-bd48-3d2e6051d75c nodeName:}" failed. No retries permitted until 2026-07-03 10:06:02.026864307 +0000 UTC m=+40.092939498 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls") pod "dns-default-bdd9n" (UID: "efd9c9dc-f958-4414-bd48-3d2e6051d75c") : secret "dns-default-metrics-tls" not found Jul 03 10:06:00.027112 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:00.026912 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:06:00.027112 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.026927 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 10:06:00.027112 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.026941 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847694c56-wkv9n: secret "image-registry-tls" not found Jul 03 10:06:00.027112 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.026971 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls podName:9faa5c7c-106a-49eb-9b2d-074de55a3bcd nodeName:}" failed. No retries permitted until 2026-07-03 10:06:02.026961059 +0000 UTC m=+40.093036232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls") pod "image-registry-5847694c56-wkv9n" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd") : secret "image-registry-tls" not found Jul 03 10:06:00.027112 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.026994 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 10:06:00.027112 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.027024 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls podName:6eac509e-8324-4a69-ae4d-7923496d16a0 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:02.027014802 +0000 UTC m=+40.093089973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c4dj5" (UID: "6eac509e-8324-4a69-ae4d-7923496d16a0") : secret "cluster-monitoring-operator-tls" not found Jul 03 10:06:00.027112 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.027072 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 10:06:00.027439 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.027125 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls podName:3114663a-1656-4dc5-9f4d-288871787eff nodeName:}" failed. No retries permitted until 2026-07-03 10:06:02.02711501 +0000 UTC m=+40.093190195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8d988" (UID: "3114663a-1656-4dc5-9f4d-288871787eff") : secret "samples-operator-tls" not found Jul 03 10:06:00.027439 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.027172 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 03 10:06:00.027439 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:00.027201 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:02.027192454 +0000 UTC m=+40.093267631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : secret "router-metrics-certs-default" not found Jul 03 10:06:01.657961 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:01.657702 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" event={"ID":"8cc555a9-6421-474e-bcb7-399b574dc972","Type":"ContainerStarted","Data":"413bb90a4e238c93f36d9678de9596111d17d01bc1b109b9d0def3e3ce256687"} Jul 03 10:06:01.663716 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:01.663681 2577 generic.go:356] "Generic (PLEG): container finished" podID="3a89dafb-6d42-40f3-b9d4-cb4a0c839af6" containerID="bbd284177ef67601a83a2b19b935c6fca516c473f33b0b1ff9e342ce346fbd26" exitCode=0 Jul 03 10:06:01.663878 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:01.663755 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" event={"ID":"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6","Type":"ContainerDied","Data":"bbd284177ef67601a83a2b19b935c6fca516c473f33b0b1ff9e342ce346fbd26"} Jul 03 10:06:02.050571 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:02.050484 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:06:02.050571 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:02.050550 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:02.050807 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:02.050628 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:06:02.050807 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:02.050656 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:06:02.050807 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:02.050711 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:02.050807 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:02.050757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:06:02.051041 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.050901 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 10:06:02.051041 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.050920 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847694c56-wkv9n: secret "image-registry-tls" not found Jul 03 10:06:02.051041 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.050981 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls podName:9faa5c7c-106a-49eb-9b2d-074de55a3bcd nodeName:}" failed. No retries permitted until 2026-07-03 10:06:06.050962048 +0000 UTC m=+44.117037219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls") pod "image-registry-5847694c56-wkv9n" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd") : secret "image-registry-tls" not found Jul 03 10:06:02.051425 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.051403 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 10:06:02.051492 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.051468 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls podName:6eac509e-8324-4a69-ae4d-7923496d16a0 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:06.051451407 +0000 UTC m=+44.117526578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c4dj5" (UID: "6eac509e-8324-4a69-ae4d-7923496d16a0") : secret "cluster-monitoring-operator-tls" not found Jul 03 10:06:02.051584 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.051557 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:06.051544501 +0000 UTC m=+44.117619671 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : configmap references non-existent config key: service-ca.crt Jul 03 10:06:02.051649 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.051625 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 10:06:02.051700 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.051662 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls podName:3114663a-1656-4dc5-9f4d-288871787eff nodeName:}" failed. No retries permitted until 2026-07-03 10:06:06.051650743 +0000 UTC m=+44.117725914 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8d988" (UID: "3114663a-1656-4dc5-9f4d-288871787eff") : secret "samples-operator-tls" not found Jul 03 10:06:02.051756 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.051714 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 10:06:02.051756 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.051742 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls podName:efd9c9dc-f958-4414-bd48-3d2e6051d75c nodeName:}" failed. No retries permitted until 2026-07-03 10:06:06.051733135 +0000 UTC m=+44.117808308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls") pod "dns-default-bdd9n" (UID: "efd9c9dc-f958-4414-bd48-3d2e6051d75c") : secret "dns-default-metrics-tls" not found Jul 03 10:06:02.051854 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.051794 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 03 10:06:02.051854 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:02.051823 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:06.051813064 +0000 UTC m=+44.117888237 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : secret "router-metrics-certs-default" not found Jul 03 10:06:02.673187 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:02.673140 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" event={"ID":"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6","Type":"ContainerStarted","Data":"32c0874a3881eb35a2a02a5a2850cbecf3da5a1caed44bac14bba283002271e5"} Jul 03 10:06:03.680316 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:03.680279 2577 generic.go:356] "Generic (PLEG): container finished" podID="3a89dafb-6d42-40f3-b9d4-cb4a0c839af6" containerID="32c0874a3881eb35a2a02a5a2850cbecf3da5a1caed44bac14bba283002271e5" exitCode=0 Jul 03 10:06:03.680766 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:03.680327 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" event={"ID":"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6","Type":"ContainerDied","Data":"32c0874a3881eb35a2a02a5a2850cbecf3da5a1caed44bac14bba283002271e5"} Jul 03 10:06:06.091166 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:06.091116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:06:06.091166 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:06.091173 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:06.091229 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091236 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091299 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls podName:3114663a-1656-4dc5-9f4d-288871787eff nodeName:}" failed. No retries permitted until 2026-07-03 10:06:14.09128152 +0000 UTC m=+52.157356707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8d988" (UID: "3114663a-1656-4dc5-9f4d-288871787eff") : secret "samples-operator-tls" not found Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091318 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091352 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:14.091341919 +0000 UTC m=+52.157417103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : secret "router-metrics-certs-default" not found Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091353 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:06.091389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091414 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls podName:efd9c9dc-f958-4414-bd48-3d2e6051d75c nodeName:}" failed. No retries permitted until 2026-07-03 10:06:14.091397531 +0000 UTC m=+52.157472707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls") pod "dns-default-bdd9n" (UID: "efd9c9dc-f958-4414-bd48-3d2e6051d75c") : secret "dns-default-metrics-tls" not found Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:06.091465 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091481 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091496 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847694c56-wkv9n: secret "image-registry-tls" not found Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:06.091511 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091539 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls podName:9faa5c7c-106a-49eb-9b2d-074de55a3bcd nodeName:}" failed. No retries permitted until 2026-07-03 10:06:14.09152501 +0000 UTC m=+52.157600199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls") pod "image-registry-5847694c56-wkv9n" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd") : secret "image-registry-tls" not found Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091582 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:14.091571599 +0000 UTC m=+52.157646769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : configmap references non-existent config key: service-ca.crt Jul 03 10:06:06.091784 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091633 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 10:06:06.092486 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:06.091672 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls podName:6eac509e-8324-4a69-ae4d-7923496d16a0 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:14.091661118 +0000 UTC m=+52.157736293 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c4dj5" (UID: "6eac509e-8324-4a69-ae4d-7923496d16a0") : secret "cluster-monitoring-operator-tls" not found Jul 03 10:06:08.695661 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.695592 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" event={"ID":"41cb1751-d956-4d34-9e67-e3d994e6bf5f","Type":"ContainerStarted","Data":"aa673fa680a261b1500b91120f6898af187ee1361709ffcc32112b00bc81c625"} Jul 03 10:06:08.697263 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.697217 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr" event={"ID":"35d12807-d620-406c-ae20-d8a5ee68e92d","Type":"ContainerStarted","Data":"05af90818e6ab0fe0d087384cb914a241014e66b96ac7851c57622dc304f7313"} Jul 03 10:06:08.698821 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.698800 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/0.log" Jul 03 10:06:08.698904 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.698836 2577 generic.go:356] "Generic (PLEG): container finished" podID="8cc555a9-6421-474e-bcb7-399b574dc972" containerID="c9435c5c61405734fa477f3410d5e17580b62589b1210586baf4bf2bf002e470" exitCode=255 Jul 03 10:06:08.699062 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.699038 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" event={"ID":"8cc555a9-6421-474e-bcb7-399b574dc972","Type":"ContainerDied","Data":"c9435c5c61405734fa477f3410d5e17580b62589b1210586baf4bf2bf002e470"} Jul 03 10:06:08.699439 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.699166 2577 scope.go:117] "RemoveContainer" containerID="c9435c5c61405734fa477f3410d5e17580b62589b1210586baf4bf2bf002e470" Jul 03 10:06:08.703269 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.703244 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" event={"ID":"3a89dafb-6d42-40f3-b9d4-cb4a0c839af6","Type":"ContainerStarted","Data":"884a2f9345895d1d79b72b8305fbca180409fa5ae9dc3dbe91d6b6faa979ac21"} Jul 03 10:06:08.705261 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.705233 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" event={"ID":"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f","Type":"ContainerStarted","Data":"cd4fc3131cefe19139e1e95de566fc47d0dce17c991cdb51d87c50b58c49a286"} Jul 03 10:06:08.707109 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.707073 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-sct46" event={"ID":"7ef7157e-cc50-408f-a010-9d6d78d6ca5e","Type":"ContainerStarted","Data":"6164393739a69651334186d8e49de56fc9d194cabdec45c38efa87f5ff78596b"} Jul 03 10:06:08.708999 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.708972 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" event={"ID":"a7e396d0-1598-4e1b-8203-660c98a64ea7","Type":"ContainerStarted","Data":"1e700b7ba9f268ea8b94a227b56518cf6101161137e262c8b3ee001e9f9e7511"} Jul 03 10:06:08.710570 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.710547 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7xc64" event={"ID":"3c24a47b-93f2-4783-9f16-e2fbf02d7ceb","Type":"ContainerStarted","Data":"ff822040579ee8c549071886579983d967c39e6ab01c9d94e85039288936a485"} Jul 03 10:06:08.738217 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.738164 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" podStartSLOduration=10.628134491 podStartE2EDuration="19.738139061s" totalImagesPullingTime="9.11000457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:08.717373217 +0000 UTC m=+46.783448410" watchObservedRunningTime="2026-07-03 10:06:08.738139061 +0000 UTC m=+46.804214247" Jul 03 10:06:08.739198 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.739160 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7xc64" podStartSLOduration=1.5868959299999998 podStartE2EDuration="10.73914909s" totalImagesPullingTime="9.15225316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:08.738834438 +0000 UTC m=+46.804909611" watchObservedRunningTime="2026-07-03 10:06:08.73914909 +0000 UTC m=+46.805224286" Jul 03 10:06:08.766600 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.765646 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" podStartSLOduration=10.644669511 podStartE2EDuration="19.765623481s" totalImagesPullingTime="9.12095397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:08.765260006 +0000 UTC m=+46.831335205" watchObservedRunningTime="2026-07-03 10:06:08.765623481 +0000 UTC m=+46.831698665" Jul 03 10:06:08.786265 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.786204 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-sct46" podStartSLOduration=10.889687727 podStartE2EDuration="19.786181178s" totalImagesPullingTime="8.896493451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:08.785360633 +0000 UTC m=+46.851435824" watchObservedRunningTime="2026-07-03 10:06:08.786181178 +0000 UTC m=+46.852256371" Jul 03 10:06:08.826314 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.826248 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gcpsk" podStartSLOduration=22.796367139 podStartE2EDuration="46.826228005s" totalImagesPullingTime="24.029860866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:08.824510053 +0000 UTC m=+46.890585248" watchObservedRunningTime="2026-07-03 10:06:08.826228005 +0000 UTC m=+46.892303197" Jul 03 10:06:08.874347 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.874289 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" podStartSLOduration=10.660042829 podStartE2EDuration="19.874266399s" totalImagesPullingTime="9.21422357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:08.847616235 +0000 UTC m=+46.913691428" watchObservedRunningTime="2026-07-03 10:06:08.874266399 +0000 UTC m=+46.940341593" Jul 03 10:06:08.874628 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:08.874601 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lqvkr" podStartSLOduration=10.812271479 podStartE2EDuration="19.874591259s" totalImagesPullingTime="9.06231978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:08.872257806 +0000 UTC m=+46.938333001" watchObservedRunningTime="2026-07-03 10:06:08.874591259 +0000 UTC m=+46.940666452" Jul 03 10:06:09.716882 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.716849 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:06:09.717627 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.717604 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/0.log" Jul 03 10:06:09.717759 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.717649 2577 generic.go:356] "Generic (PLEG): container finished" podID="8cc555a9-6421-474e-bcb7-399b574dc972" containerID="ee12d4c4ce837b586f3de9568d33554b9be6cee30c79389b3d452a467e70283e" exitCode=255 Jul 03 10:06:09.717759 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.717747 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" event={"ID":"8cc555a9-6421-474e-bcb7-399b574dc972","Type":"ContainerDied","Data":"ee12d4c4ce837b586f3de9568d33554b9be6cee30c79389b3d452a467e70283e"} Jul 03 10:06:09.717871 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.717793 2577 scope.go:117] "RemoveContainer" containerID="c9435c5c61405734fa477f3410d5e17580b62589b1210586baf4bf2bf002e470" Jul 03 10:06:09.718400 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.718381 2577 scope.go:117] "RemoveContainer" containerID="ee12d4c4ce837b586f3de9568d33554b9be6cee30c79389b3d452a467e70283e" Jul 03 10:06:09.718713 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:09.718694 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-8mlv5_openshift-console-operator(8cc555a9-6421-474e-bcb7-399b574dc972)\"" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" podUID="8cc555a9-6421-474e-bcb7-399b574dc972" Jul 03 10:06:09.959212 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.959177 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kqgvb"] Jul 03 10:06:09.995045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.994975 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kqgvb"] Jul 03 10:06:09.995212 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.995135 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:09.999235 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.999210 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f8d4v\"" Jul 03 10:06:09.999455 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.999275 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 03 10:06:09.999669 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:09.999381 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 03 10:06:10.027970 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.027708 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/651ced07-9e27-481a-81a1-6df08a338b9f-data-volume\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.027970 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.027767 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/651ced07-9e27-481a-81a1-6df08a338b9f-crio-socket\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.027970 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.027800 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.027970 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.027840 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8p4b\" (UniqueName: \"kubernetes.io/projected/651ced07-9e27-481a-81a1-6df08a338b9f-kube-api-access-z8p4b\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.027970 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.027887 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.128903 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.128865 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/651ced07-9e27-481a-81a1-6df08a338b9f-data-volume\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.129122 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.128911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/651ced07-9e27-481a-81a1-6df08a338b9f-crio-socket\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.129122 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.128935 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.129122 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.128951 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8p4b\" (UniqueName: \"kubernetes.io/projected/651ced07-9e27-481a-81a1-6df08a338b9f-kube-api-access-z8p4b\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.129122 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.128982 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.129122 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:10.129039 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 10:06:10.129363 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:10.129151 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm podName:651ced07-9e27-481a-81a1-6df08a338b9f nodeName:}" failed. No retries permitted until 2026-07-03 10:06:10.629128811 +0000 UTC m=+48.695204001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kqgvb" (UID: "651ced07-9e27-481a-81a1-6df08a338b9f") : configmap "kube-rbac-proxy" not found Jul 03 10:06:10.129363 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:10.129166 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 10:06:10.129363 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.129171 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/651ced07-9e27-481a-81a1-6df08a338b9f-crio-socket\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.129363 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:10.129227 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls podName:651ced07-9e27-481a-81a1-6df08a338b9f nodeName:}" failed. No retries permitted until 2026-07-03 10:06:10.629209839 +0000 UTC m=+48.695285024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kqgvb" (UID: "651ced07-9e27-481a-81a1-6df08a338b9f") : secret "insights-runtime-extractor-tls" not found Jul 03 10:06:10.129363 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.129295 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/651ced07-9e27-481a-81a1-6df08a338b9f-data-volume\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.148395 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.148365 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8p4b\" (UniqueName: \"kubernetes.io/projected/651ced07-9e27-481a-81a1-6df08a338b9f-kube-api-access-z8p4b\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.633935 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.633894 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.634109 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:10.634060 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 10:06:10.634109 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.634078 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:10.634198 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:10.634145 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls podName:651ced07-9e27-481a-81a1-6df08a338b9f nodeName:}" failed. No retries permitted until 2026-07-03 10:06:11.634124974 +0000 UTC m=+49.700200163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kqgvb" (UID: "651ced07-9e27-481a-81a1-6df08a338b9f") : secret "insights-runtime-extractor-tls" not found Jul 03 10:06:10.634198 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:10.634167 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 10:06:10.634282 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:10.634205 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm podName:651ced07-9e27-481a-81a1-6df08a338b9f nodeName:}" failed. No retries permitted until 2026-07-03 10:06:11.634193023 +0000 UTC m=+49.700268202 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kqgvb" (UID: "651ced07-9e27-481a-81a1-6df08a338b9f") : configmap "kube-rbac-proxy" not found Jul 03 10:06:10.722338 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.722311 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:06:10.722707 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:10.722692 2577 scope.go:117] "RemoveContainer" containerID="ee12d4c4ce837b586f3de9568d33554b9be6cee30c79389b3d452a467e70283e" Jul 03 10:06:10.722869 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:10.722849 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-8mlv5_openshift-console-operator(8cc555a9-6421-474e-bcb7-399b574dc972)\"" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" podUID="8cc555a9-6421-474e-bcb7-399b574dc972" Jul 03 10:06:11.642622 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:11.642589 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:11.642806 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:11.642638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:11.642806 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:11.642729 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 10:06:11.642806 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:11.642776 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 10:06:11.642903 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:11.642813 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm podName:651ced07-9e27-481a-81a1-6df08a338b9f nodeName:}" failed. No retries permitted until 2026-07-03 10:06:13.642792617 +0000 UTC m=+51.708867791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kqgvb" (UID: "651ced07-9e27-481a-81a1-6df08a338b9f") : configmap "kube-rbac-proxy" not found Jul 03 10:06:11.642903 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:11.642832 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls podName:651ced07-9e27-481a-81a1-6df08a338b9f nodeName:}" failed. No retries permitted until 2026-07-03 10:06:13.642823003 +0000 UTC m=+51.708898176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kqgvb" (UID: "651ced07-9e27-481a-81a1-6df08a338b9f") : secret "insights-runtime-extractor-tls" not found Jul 03 10:06:11.760541 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:11.760513 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kzm94_75626273-af77-4838-99cc-2d3b274069d4/dns-node-resolver/0.log" Jul 03 10:06:12.560577 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:12.560552 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bm5md_01500b78-d50b-467c-bf74-f4b3e310e159/node-ca/0.log" Jul 03 10:06:13.164284 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:13.164253 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7xc64_3c24a47b-93f2-4783-9f16-e2fbf02d7ceb/serve-healthcheck-canary/0.log" Jul 03 10:06:13.660861 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:13.660826 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:13.661035 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:13.660878 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:13.661035 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:13.660999 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 10:06:13.661131 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:13.661062 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls podName:651ced07-9e27-481a-81a1-6df08a338b9f nodeName:}" failed. No retries permitted until 2026-07-03 10:06:17.661045348 +0000 UTC m=+55.727120517 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kqgvb" (UID: "651ced07-9e27-481a-81a1-6df08a338b9f") : secret "insights-runtime-extractor-tls" not found Jul 03 10:06:13.661399 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:13.661380 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/651ced07-9e27-481a-81a1-6df08a338b9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:14.166153 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:14.166114 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:06:14.166153 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:14.166158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:14.166198 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:14.166230 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:14.166261 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166280 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:14.166286 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166360 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166371 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166379 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls podName:efd9c9dc-f958-4414-bd48-3d2e6051d75c nodeName:}" failed. No retries permitted until 2026-07-03 10:06:30.166352177 +0000 UTC m=+68.232427352 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls") pod "dns-default-bdd9n" (UID: "efd9c9dc-f958-4414-bd48-3d2e6051d75c") : secret "dns-default-metrics-tls" not found Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166280 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166405 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls podName:6eac509e-8324-4a69-ae4d-7923496d16a0 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:30.166394468 +0000 UTC m=+68.232469661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c4dj5" (UID: "6eac509e-8324-4a69-ae4d-7923496d16a0") : secret "cluster-monitoring-operator-tls" not found Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166432 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls podName:3114663a-1656-4dc5-9f4d-288871787eff nodeName:}" failed. No retries permitted until 2026-07-03 10:06:30.166424756 +0000 UTC m=+68.232499925 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8d988" (UID: "3114663a-1656-4dc5-9f4d-288871787eff") : secret "samples-operator-tls" not found Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166445 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:30.166439462 +0000 UTC m=+68.232514632 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : configmap references non-existent config key: service-ca.crt Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166455 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs podName:7a741adb-6089-40c5-ba07-7f334eadd790 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:30.166450331 +0000 UTC m=+68.232525501 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs") pod "router-default-7f7ff5477c-6gn75" (UID: "7a741adb-6089-40c5-ba07-7f334eadd790") : secret "router-metrics-certs-default" not found Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166473 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 10:06:14.166651 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166491 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847694c56-wkv9n: secret "image-registry-tls" not found Jul 03 10:06:14.167130 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:14.166533 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls podName:9faa5c7c-106a-49eb-9b2d-074de55a3bcd nodeName:}" failed. No retries permitted until 2026-07-03 10:06:30.166517004 +0000 UTC m=+68.232592174 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls") pod "image-registry-5847694c56-wkv9n" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd") : secret "image-registry-tls" not found Jul 03 10:06:17.696895 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:17.696852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:17.697595 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:17.697573 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 10:06:17.697683 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:17.697665 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls podName:651ced07-9e27-481a-81a1-6df08a338b9f nodeName:}" failed. No retries permitted until 2026-07-03 10:06:25.69764367 +0000 UTC m=+63.763718862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kqgvb" (UID: "651ced07-9e27-481a-81a1-6df08a338b9f") : secret "insights-runtime-extractor-tls" not found Jul 03 10:06:18.658854 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:18.658808 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:06:18.659034 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:18.658891 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:06:18.659280 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:18.659267 2577 scope.go:117] "RemoveContainer" containerID="ee12d4c4ce837b586f3de9568d33554b9be6cee30c79389b3d452a467e70283e" Jul 03 10:06:18.659436 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:06:18.659418 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-8mlv5_openshift-console-operator(8cc555a9-6421-474e-bcb7-399b574dc972)\"" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" podUID="8cc555a9-6421-474e-bcb7-399b574dc972" Jul 03 10:06:23.639313 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:23.639284 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hb8xm" Jul 03 10:06:25.772872 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:25.772837 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:25.775151 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:25.775119 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/651ced07-9e27-481a-81a1-6df08a338b9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqgvb\" (UID: \"651ced07-9e27-481a-81a1-6df08a338b9f\") " pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:25.909107 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:25.909072 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f8d4v\"" Jul 03 10:06:25.916481 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:25.916461 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kqgvb" Jul 03 10:06:26.041308 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:26.041261 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kqgvb"] Jul 03 10:06:26.043758 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:26.043734 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod651ced07_9e27_481a_81a1_6df08a338b9f.slice/crio-1d8012d3cde5efa7a0026c0bff0c3543b7f0222ab97bbea01be230ad053e85a0 WatchSource:0}: Error finding container 1d8012d3cde5efa7a0026c0bff0c3543b7f0222ab97bbea01be230ad053e85a0: Status 404 returned error can't find the container with id 1d8012d3cde5efa7a0026c0bff0c3543b7f0222ab97bbea01be230ad053e85a0 Jul 03 10:06:26.772961 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:26.772919 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kqgvb" event={"ID":"651ced07-9e27-481a-81a1-6df08a338b9f","Type":"ContainerStarted","Data":"1d8012d3cde5efa7a0026c0bff0c3543b7f0222ab97bbea01be230ad053e85a0"} Jul 03 10:06:28.294944 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.294890 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:06:28.295433 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.294950 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkl8\" (UniqueName: \"kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8\") pod \"network-check-target-hkgr6\" (UID: \"12d45bd8-f937-4c62-98a4-63a18defb862\") " pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:06:28.295433 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.295082 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:06:28.297790 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.297764 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqkl8\" (UniqueName: \"kubernetes.io/projected/12d45bd8-f937-4c62-98a4-63a18defb862-kube-api-access-cqkl8\") pod \"network-check-target-hkgr6\" (UID: \"12d45bd8-f937-4c62-98a4-63a18defb862\") " pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:06:28.298172 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.298150 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 10:06:28.299339 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.299316 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 10:06:28.308460 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.308433 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/108d4f92-ca93-47d8-a45f-ba7088301bfa-original-pull-secret\") pod \"global-pull-secret-syncer-vz6gj\" (UID: \"108d4f92-ca93-47d8-a45f-ba7088301bfa\") " pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:06:28.308575 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.308459 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad84ff02-cefd-4c8e-b897-8a97a3727b79-metrics-certs\") pod \"network-metrics-daemon-2rjc5\" (UID: \"ad84ff02-cefd-4c8e-b897-8a97a3727b79\") " pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:06:28.480149 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.480088 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xh5d7\"" Jul 03 10:06:28.485039 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.485010 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-njhrs\"" Jul 03 10:06:28.487163 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.487139 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2rjc5" Jul 03 10:06:28.487301 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.487170 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6gj" Jul 03 10:06:28.492530 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:28.492508 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:06:29.466965 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:29.466722 2577 scope.go:117] "RemoveContainer" containerID="ee12d4c4ce837b586f3de9568d33554b9be6cee30c79389b3d452a467e70283e" Jul 03 10:06:29.553061 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:29.553009 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2rjc5"] Jul 03 10:06:29.777073 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:29.776939 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad84ff02_cefd_4c8e_b897_8a97a3727b79.slice/crio-71147c57edabb35b564c2b340fdd80a5af4bb093bfed10220ccda96f46363f8c WatchSource:0}: Error finding container 71147c57edabb35b564c2b340fdd80a5af4bb093bfed10220ccda96f46363f8c: Status 404 returned error can't find the container with id 71147c57edabb35b564c2b340fdd80a5af4bb093bfed10220ccda96f46363f8c Jul 03 10:06:29.787277 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:29.787240 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2rjc5" event={"ID":"ad84ff02-cefd-4c8e-b897-8a97a3727b79","Type":"ContainerStarted","Data":"71147c57edabb35b564c2b340fdd80a5af4bb093bfed10220ccda96f46363f8c"} Jul 03 10:06:29.921128 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:29.921001 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vz6gj"] Jul 03 10:06:29.923824 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:29.922996 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108d4f92_ca93_47d8_a45f_ba7088301bfa.slice/crio-6d652590a20b5d14b767f432cf25b23d420ae9607930ca18a313ad3693f251a4 WatchSource:0}: Error finding container 6d652590a20b5d14b767f432cf25b23d420ae9607930ca18a313ad3693f251a4: Status 404 returned error can't find the container with id 6d652590a20b5d14b767f432cf25b23d420ae9607930ca18a313ad3693f251a4 Jul 03 10:06:29.939637 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:29.939602 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hkgr6"] Jul 03 10:06:29.942332 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:29.942302 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d45bd8_f937_4c62_98a4_63a18defb862.slice/crio-bf37325b592ff306b815cc6c361e522713401978c416e188f132d12032f77375 WatchSource:0}: Error finding container bf37325b592ff306b815cc6c361e522713401978c416e188f132d12032f77375: Status 404 returned error can't find the container with id bf37325b592ff306b815cc6c361e522713401978c416e188f132d12032f77375 Jul 03 10:06:30.213013 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.212971 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:30.213247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.213036 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:06:30.213247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.213061 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:06:30.213247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.213086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:30.213247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.213141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:06:30.213247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.213168 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:06:30.213796 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.213773 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a741adb-6089-40c5-ba07-7f334eadd790-service-ca-bundle\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:30.216049 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.216022 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") pod \"image-registry-5847694c56-wkv9n\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:06:30.216169 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.216063 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a741adb-6089-40c5-ba07-7f334eadd790-metrics-certs\") pod \"router-default-7f7ff5477c-6gn75\" (UID: \"7a741adb-6089-40c5-ba07-7f334eadd790\") " pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:30.216169 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.216063 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efd9c9dc-f958-4414-bd48-3d2e6051d75c-metrics-tls\") pod \"dns-default-bdd9n\" (UID: \"efd9c9dc-f958-4414-bd48-3d2e6051d75c\") " pod="openshift-dns/dns-default-bdd9n" Jul 03 10:06:30.216169 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.216132 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eac509e-8324-4a69-ae4d-7923496d16a0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c4dj5\" (UID: \"6eac509e-8324-4a69-ae4d-7923496d16a0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:06:30.216169 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.216157 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3114663a-1656-4dc5-9f4d-288871787eff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8d988\" (UID: \"3114663a-1656-4dc5-9f4d-288871787eff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:06:30.370079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.370048 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zmbwq\"" Jul 03 10:06:30.377933 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.377900 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:06:30.395194 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.395166 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-g8qr5\"" Jul 03 10:06:30.402812 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.402791 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" Jul 03 10:06:30.408760 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.408740 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r2dls\"" Jul 03 10:06:30.416130 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.416090 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" Jul 03 10:06:30.423694 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.423672 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zg8hw\"" Jul 03 10:06:30.432942 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.432177 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:30.437658 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.436413 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sgbfl\"" Jul 03 10:06:30.443478 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.440376 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bdd9n" Jul 03 10:06:30.599571 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.599182 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5847694c56-wkv9n"] Jul 03 10:06:30.640879 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.640805 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988"] Jul 03 10:06:30.708207 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.706957 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5"] Jul 03 10:06:30.712117 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:30.711986 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eac509e_8324_4a69_ae4d_7923496d16a0.slice/crio-27979e29e087c3f1b63479ab8e42568fed2f9ae5a9313a9cc4f8e89056c17922 WatchSource:0}: Error finding container 27979e29e087c3f1b63479ab8e42568fed2f9ae5a9313a9cc4f8e89056c17922: Status 404 returned error can't find the container with id 27979e29e087c3f1b63479ab8e42568fed2f9ae5a9313a9cc4f8e89056c17922 Jul 03 10:06:30.728008 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.727985 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f7ff5477c-6gn75"] Jul 03 10:06:30.735863 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:30.735839 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a741adb_6089_40c5_ba07_7f334eadd790.slice/crio-743a7981b54bc798a4f059110d6a7606e900674f4a527f7d73e7c0ab74916931 WatchSource:0}: Error finding container 743a7981b54bc798a4f059110d6a7606e900674f4a527f7d73e7c0ab74916931: Status 404 returned error can't find the container with id 743a7981b54bc798a4f059110d6a7606e900674f4a527f7d73e7c0ab74916931 Jul 03 10:06:30.747419 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.747394 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bdd9n"] Jul 03 10:06:30.753420 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:30.753389 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd9c9dc_f958_4414_bd48_3d2e6051d75c.slice/crio-0fc56521647ff89c8d59ff6350c7d6e065acbaefed046aff24da76bfb50b4530 WatchSource:0}: Error finding container 0fc56521647ff89c8d59ff6350c7d6e065acbaefed046aff24da76bfb50b4530: Status 404 returned error can't find the container with id 0fc56521647ff89c8d59ff6350c7d6e065acbaefed046aff24da76bfb50b4530 Jul 03 10:06:30.800870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.800753 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kqgvb" event={"ID":"651ced07-9e27-481a-81a1-6df08a338b9f","Type":"ContainerStarted","Data":"5663f1c77ee93599606261fa383148af4dbb6b64cebd1d27df27c7601af9ca8b"} Jul 03 10:06:30.800870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.800823 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kqgvb" event={"ID":"651ced07-9e27-481a-81a1-6df08a338b9f","Type":"ContainerStarted","Data":"00fea3952bcad1bc05e5b951bf99224f6eca4fefd950f1a70481ea5446b4a767"} Jul 03 10:06:30.808590 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.808551 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f7ff5477c-6gn75" event={"ID":"7a741adb-6089-40c5-ba07-7f334eadd790","Type":"ContainerStarted","Data":"743a7981b54bc798a4f059110d6a7606e900674f4a527f7d73e7c0ab74916931"} Jul 03 10:06:30.811368 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.811329 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bdd9n" event={"ID":"efd9c9dc-f958-4414-bd48-3d2e6051d75c","Type":"ContainerStarted","Data":"0fc56521647ff89c8d59ff6350c7d6e065acbaefed046aff24da76bfb50b4530"} Jul 03 10:06:30.813064 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.813035 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" event={"ID":"3114663a-1656-4dc5-9f4d-288871787eff","Type":"ContainerStarted","Data":"a1efd41032e15e69eeb77c8e65f6670cbbda3a4d0e6b705074af9d42a8d24cad"} Jul 03 10:06:30.814728 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.814705 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" event={"ID":"9faa5c7c-106a-49eb-9b2d-074de55a3bcd","Type":"ContainerStarted","Data":"a5868b2c322a612eabb63a080a46ee20be4a7d64f33ee2cc90c44e7f20191594"} Jul 03 10:06:30.814865 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.814850 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" event={"ID":"9faa5c7c-106a-49eb-9b2d-074de55a3bcd","Type":"ContainerStarted","Data":"38426428e16253d56259b2dc2a5088bde6933b4be068f2f8c4ab4d803235e1f7"} Jul 03 10:06:30.815171 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.815151 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:06:30.818075 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.817407 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hkgr6" event={"ID":"12d45bd8-f937-4c62-98a4-63a18defb862","Type":"ContainerStarted","Data":"f9da9b3bf1a8b606f659c8d957bb3ef99ff98b307a1c0ac0b08da8dd8b40a03e"} Jul 03 10:06:30.818075 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.817461 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hkgr6" event={"ID":"12d45bd8-f937-4c62-98a4-63a18defb862","Type":"ContainerStarted","Data":"bf37325b592ff306b815cc6c361e522713401978c416e188f132d12032f77375"} Jul 03 10:06:30.818075 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.817973 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:06:30.824527 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.824509 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:06:30.825166 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.825138 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" event={"ID":"8cc555a9-6421-474e-bcb7-399b574dc972","Type":"ContainerStarted","Data":"d02d321bdc4113353988f742559686ced226a83dfabf396a1bcf7ab8d124eb2b"} Jul 03 10:06:30.825446 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.825433 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:06:30.828271 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.828246 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" event={"ID":"6eac509e-8324-4a69-ae4d-7923496d16a0","Type":"ContainerStarted","Data":"27979e29e087c3f1b63479ab8e42568fed2f9ae5a9313a9cc4f8e89056c17922"} Jul 03 10:06:30.829631 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.829607 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vz6gj" event={"ID":"108d4f92-ca93-47d8-a45f-ba7088301bfa","Type":"ContainerStarted","Data":"6d652590a20b5d14b767f432cf25b23d420ae9607930ca18a313ad3693f251a4"} Jul 03 10:06:30.838741 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.838582 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" podStartSLOduration=68.838564189 podStartE2EDuration="1m8.838564189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:30.837747283 +0000 UTC m=+68.903822477" watchObservedRunningTime="2026-07-03 10:06:30.838564189 +0000 UTC m=+68.904639381" Jul 03 10:06:30.855948 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.855867 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hkgr6" podStartSLOduration=68.855846965 podStartE2EDuration="1m8.855846965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:30.854428158 +0000 UTC m=+68.920503376" watchObservedRunningTime="2026-07-03 10:06:30.855846965 +0000 UTC m=+68.921922158" Jul 03 10:06:30.861301 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.861265 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" Jul 03 10:06:30.875826 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:30.875330 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-8mlv5" podStartSLOduration=34.492634103 podStartE2EDuration="41.875308782s" totalImagesPullingTime="7.382674679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:30.874727113 +0000 UTC m=+68.940802306" watchObservedRunningTime="2026-07-03 10:06:30.875308782 +0000 UTC m=+68.941383974" Jul 03 10:06:31.837211 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:31.837124 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f7ff5477c-6gn75" event={"ID":"7a741adb-6089-40c5-ba07-7f334eadd790","Type":"ContainerStarted","Data":"5fef671f8d62b0308d9552f0737185b3f23ae789bee734ccc0c08c46da3bfaa3"} Jul 03 10:06:31.878899 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:31.877496 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7f7ff5477c-6gn75" podStartSLOduration=42.877471616 podStartE2EDuration="42.877471616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:31.876800105 +0000 UTC m=+69.942875318" watchObservedRunningTime="2026-07-03 10:06:31.877471616 +0000 UTC m=+69.943546810" Jul 03 10:06:32.433511 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:32.433481 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:32.436641 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:32.436605 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:32.852878 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:32.852541 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2rjc5" event={"ID":"ad84ff02-cefd-4c8e-b897-8a97a3727b79","Type":"ContainerStarted","Data":"4caf29e53ee9dc827cf6f401eb438285f122d37698b02d070b766809eb0f86aa"} Jul 03 10:06:32.852878 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:32.852641 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2rjc5" event={"ID":"ad84ff02-cefd-4c8e-b897-8a97a3727b79","Type":"ContainerStarted","Data":"01cfd82202ceb640dc5fa22889597870e6faadb5bdb04abcbec968264d8a10fe"} Jul 03 10:06:32.853698 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:32.853522 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:32.854774 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:32.854735 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7f7ff5477c-6gn75" Jul 03 10:06:32.871164 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:32.871083 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2rjc5" podStartSLOduration=68.819669517 podStartE2EDuration="1m10.871064848s" totalImagesPullingTime="2.051395331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:32.87061661 +0000 UTC m=+70.936691804" watchObservedRunningTime="2026-07-03 10:06:32.871064848 +0000 UTC m=+70.937140041" Jul 03 10:06:36.871272 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:36.871237 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kqgvb" event={"ID":"651ced07-9e27-481a-81a1-6df08a338b9f","Type":"ContainerStarted","Data":"deabe25ea6e4bfac56d709399e5beb7533867eee14b52718d317c8bfb52b3a14"} Jul 03 10:06:36.874944 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:36.874911 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bdd9n" event={"ID":"efd9c9dc-f958-4414-bd48-3d2e6051d75c","Type":"ContainerStarted","Data":"5c7a77f0cff9e4c47ef075a813b8df7e56659d797d9000392a887bbe63ece192"} Jul 03 10:06:36.876583 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:36.876556 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" event={"ID":"3114663a-1656-4dc5-9f4d-288871787eff","Type":"ContainerStarted","Data":"9f8983355abb1a1ef14619bd666339d3d570912602f8cfb1b707b72cacd85c96"} Jul 03 10:06:36.888855 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:36.888375 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" event={"ID":"6eac509e-8324-4a69-ae4d-7923496d16a0","Type":"ContainerStarted","Data":"7e6d232f4a3780a7e65d55673cafb6ad3bbdbead5b1168f3ad4ca5cb17980186"} Jul 03 10:06:36.933493 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:36.932054 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kqgvb" podStartSLOduration=17.476518332 podStartE2EDuration="27.932030101s" totalImagesPullingTime="10.455511769s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:06:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:36.909638812 +0000 UTC m=+74.975714005" watchObservedRunningTime="2026-07-03 10:06:36.932030101 +0000 UTC m=+74.998105294" Jul 03 10:06:36.933493 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:36.932875 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c4dj5" podStartSLOduration=41.898370418 podStartE2EDuration="47.932859484s" totalImagesPullingTime="6.034489066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:36.932424195 +0000 UTC m=+74.998499387" watchObservedRunningTime="2026-07-03 10:06:36.932859484 +0000 UTC m=+74.998934676" Jul 03 10:06:37.499733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.499699 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4"] Jul 03 10:06:37.503325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.503303 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.507293 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.507265 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 03 10:06:37.507487 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.507466 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 03 10:06:37.507627 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.507384 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 03 10:06:37.507977 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.507962 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 03 10:06:37.516294 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.516262 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4"] Jul 03 10:06:37.584916 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.584887 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbnrl\" (UniqueName: \"kubernetes.io/projected/8ee34c3b-77d9-4c6d-9ffb-99c350fbe353-kube-api-access-lbnrl\") pod \"klusterlet-addon-workmgr-869b645f9d-q94x4\" (UID: \"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.585080 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.584942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ee34c3b-77d9-4c6d-9ffb-99c350fbe353-tmp\") pod \"klusterlet-addon-workmgr-869b645f9d-q94x4\" (UID: \"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.585080 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.585019 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8ee34c3b-77d9-4c6d-9ffb-99c350fbe353-klusterlet-config\") pod \"klusterlet-addon-workmgr-869b645f9d-q94x4\" (UID: \"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.596045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.594756 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-spvxp"] Jul 03 10:06:37.598373 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.598339 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85"] Jul 03 10:06:37.598540 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.598471 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-spvxp" Jul 03 10:06:37.601450 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.601422 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 03 10:06:37.601637 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.601456 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 03 10:06:37.601637 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.601546 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85" Jul 03 10:06:37.602030 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.602007 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7q5vc\"" Jul 03 10:06:37.603985 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.603967 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 03 10:06:37.604684 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.604671 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8lff2\"" Jul 03 10:06:37.613526 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.613502 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-spvxp"] Jul 03 10:06:37.630650 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.630624 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85"] Jul 03 10:06:37.686127 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.686071 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f5df1fbc-077d-4cd4-a01a-c081349e9c25-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-dsc85\" (UID: \"f5df1fbc-077d-4cd4-a01a-c081349e9c25\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85" Jul 03 10:06:37.686304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.686152 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ee34c3b-77d9-4c6d-9ffb-99c350fbe353-tmp\") pod \"klusterlet-addon-workmgr-869b645f9d-q94x4\" (UID: \"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.686304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.686212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8ee34c3b-77d9-4c6d-9ffb-99c350fbe353-klusterlet-config\") pod \"klusterlet-addon-workmgr-869b645f9d-q94x4\" (UID: \"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.686304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.686256 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjdcr\" (UniqueName: \"kubernetes.io/projected/a54488f8-1418-4d53-ace8-1120c1c8fe30-kube-api-access-hjdcr\") pod \"downloads-685bfb5f64-spvxp\" (UID: \"a54488f8-1418-4d53-ace8-1120c1c8fe30\") " pod="openshift-console/downloads-685bfb5f64-spvxp" Jul 03 10:06:37.686415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.686304 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbnrl\" (UniqueName: \"kubernetes.io/projected/8ee34c3b-77d9-4c6d-9ffb-99c350fbe353-kube-api-access-lbnrl\") pod \"klusterlet-addon-workmgr-869b645f9d-q94x4\" (UID: \"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.686503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.686486 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ee34c3b-77d9-4c6d-9ffb-99c350fbe353-tmp\") pod \"klusterlet-addon-workmgr-869b645f9d-q94x4\" (UID: \"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.688674 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.688653 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8ee34c3b-77d9-4c6d-9ffb-99c350fbe353-klusterlet-config\") pod \"klusterlet-addon-workmgr-869b645f9d-q94x4\" (UID: \"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.701124 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.701067 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbnrl\" (UniqueName: \"kubernetes.io/projected/8ee34c3b-77d9-4c6d-9ffb-99c350fbe353-kube-api-access-lbnrl\") pod \"klusterlet-addon-workmgr-869b645f9d-q94x4\" (UID: \"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.787307 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.787209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f5df1fbc-077d-4cd4-a01a-c081349e9c25-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-dsc85\" (UID: \"f5df1fbc-077d-4cd4-a01a-c081349e9c25\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85" Jul 03 10:06:37.787307 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.787286 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjdcr\" (UniqueName: \"kubernetes.io/projected/a54488f8-1418-4d53-ace8-1120c1c8fe30-kube-api-access-hjdcr\") pod \"downloads-685bfb5f64-spvxp\" (UID: \"a54488f8-1418-4d53-ace8-1120c1c8fe30\") " pod="openshift-console/downloads-685bfb5f64-spvxp" Jul 03 10:06:37.789824 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.789793 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f5df1fbc-077d-4cd4-a01a-c081349e9c25-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-dsc85\" (UID: \"f5df1fbc-077d-4cd4-a01a-c081349e9c25\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85" Jul 03 10:06:37.804871 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.804828 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjdcr\" (UniqueName: \"kubernetes.io/projected/a54488f8-1418-4d53-ace8-1120c1c8fe30-kube-api-access-hjdcr\") pod \"downloads-685bfb5f64-spvxp\" (UID: \"a54488f8-1418-4d53-ace8-1120c1c8fe30\") " pod="openshift-console/downloads-685bfb5f64-spvxp" Jul 03 10:06:37.816673 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.816637 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:37.894734 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.894695 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vz6gj" event={"ID":"108d4f92-ca93-47d8-a45f-ba7088301bfa","Type":"ContainerStarted","Data":"02618fd6182ece3eec75a9c2cb4233cbf4def2136aa3bc017321ea1563676ee8"} Jul 03 10:06:37.897076 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.897049 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bdd9n" event={"ID":"efd9c9dc-f958-4414-bd48-3d2e6051d75c","Type":"ContainerStarted","Data":"fd5c58e9f69f056d072a7d615cb71b7fed2e77d29396270f88b28edcf87eae36"} Jul 03 10:06:37.897342 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.897288 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bdd9n" Jul 03 10:06:37.899189 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.899163 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" event={"ID":"3114663a-1656-4dc5-9f4d-288871787eff","Type":"ContainerStarted","Data":"61c70fe1b99149c99bbb4931667acca7bbbead33f435928cc886e34c85f001c6"} Jul 03 10:06:37.912497 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.912417 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-spvxp" Jul 03 10:06:37.917508 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.917477 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85" Jul 03 10:06:37.921424 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.921341 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vz6gj" podStartSLOduration=69.082712849 podStartE2EDuration="1m15.921318845s" totalImagesPullingTime="6.838605996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:37.920322381 +0000 UTC m=+75.986397576" watchObservedRunningTime="2026-07-03 10:06:37.921318845 +0000 UTC m=+75.987394041" Jul 03 10:06:37.960036 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.959981 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bdd9n" podStartSLOduration=33.966709367 podStartE2EDuration="39.959961277s" totalImagesPullingTime="5.99325191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:37.959001693 +0000 UTC m=+76.025076887" watchObservedRunningTime="2026-07-03 10:06:37.959961277 +0000 UTC m=+76.026036519" Jul 03 10:06:37.973999 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.973963 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4"] Jul 03 10:06:37.977357 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:37.977316 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee34c3b_77d9_4c6d_9ffb_99c350fbe353.slice/crio-54f1955a677e2ef31b82a686b7f7028d3d9087a3bee8a7fd3940ec63cc5d9930 WatchSource:0}: Error finding container 54f1955a677e2ef31b82a686b7f7028d3d9087a3bee8a7fd3940ec63cc5d9930: Status 404 returned error can't find the container with id 54f1955a677e2ef31b82a686b7f7028d3d9087a3bee8a7fd3940ec63cc5d9930 Jul 03 10:06:37.988788 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:37.987456 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8d988" podStartSLOduration=42.954465383 podStartE2EDuration="48.987434314s" totalImagesPullingTime="6.032968931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:37.985546316 +0000 UTC m=+76.051621509" watchObservedRunningTime="2026-07-03 10:06:37.987434314 +0000 UTC m=+76.053509511" Jul 03 10:06:38.080967 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:38.080944 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-spvxp"] Jul 03 10:06:38.082072 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:38.082052 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda54488f8_1418_4d53_ace8_1120c1c8fe30.slice/crio-99ca7190365b4bb910a6a0e6f71a08920286a3b5b8107b2f659daa9386cc38e9 WatchSource:0}: Error finding container 99ca7190365b4bb910a6a0e6f71a08920286a3b5b8107b2f659daa9386cc38e9: Status 404 returned error can't find the container with id 99ca7190365b4bb910a6a0e6f71a08920286a3b5b8107b2f659daa9386cc38e9 Jul 03 10:06:38.087487 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:38.087463 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85"] Jul 03 10:06:38.090186 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:38.090163 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5df1fbc_077d_4cd4_a01a_c081349e9c25.slice/crio-d0979277346864af32381ebc54f79b7c7eb2a2baa3ab3d919ebcf3c21628df26 WatchSource:0}: Error finding container d0979277346864af32381ebc54f79b7c7eb2a2baa3ab3d919ebcf3c21628df26: Status 404 returned error can't find the container with id d0979277346864af32381ebc54f79b7c7eb2a2baa3ab3d919ebcf3c21628df26 Jul 03 10:06:38.908604 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:38.908567 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85" event={"ID":"f5df1fbc-077d-4cd4-a01a-c081349e9c25","Type":"ContainerStarted","Data":"d0979277346864af32381ebc54f79b7c7eb2a2baa3ab3d919ebcf3c21628df26"} Jul 03 10:06:38.910421 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:38.910387 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" event={"ID":"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353","Type":"ContainerStarted","Data":"54f1955a677e2ef31b82a686b7f7028d3d9087a3bee8a7fd3940ec63cc5d9930"} Jul 03 10:06:38.911901 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:38.911870 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-spvxp" event={"ID":"a54488f8-1418-4d53-ace8-1120c1c8fe30","Type":"ContainerStarted","Data":"99ca7190365b4bb910a6a0e6f71a08920286a3b5b8107b2f659daa9386cc38e9"} Jul 03 10:06:39.427577 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.425547 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c866df8d5-llmzc"] Jul 03 10:06:39.442440 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.442400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.445876 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.445802 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c866df8d5-llmzc"] Jul 03 10:06:39.446854 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.446437 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b5rfb\"" Jul 03 10:06:39.446854 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.446541 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 03 10:06:39.448502 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.448431 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 03 10:06:39.448719 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.448698 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 03 10:06:39.450275 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.448896 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 03 10:06:39.450275 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.449143 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 03 10:06:39.603884 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.603732 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-oauth-serving-cert\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.603884 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.603824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gfld\" (UniqueName: \"kubernetes.io/projected/c215faef-0b04-4f9c-b03d-c77824db5d43-kube-api-access-7gfld\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.603884 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.603854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-serving-cert\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.604244 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.603914 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-oauth-config\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.604244 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.603993 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-console-config\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.604244 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.604022 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-service-ca\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.705385 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.705282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gfld\" (UniqueName: \"kubernetes.io/projected/c215faef-0b04-4f9c-b03d-c77824db5d43-kube-api-access-7gfld\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.705385 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.705338 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-serving-cert\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.705385 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.705385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-oauth-config\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.705726 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.705431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-console-config\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.705726 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.705460 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-service-ca\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.705726 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.705493 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-oauth-serving-cert\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.706983 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.706956 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-console-config\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.707675 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.707628 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-oauth-serving-cert\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.707789 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.707695 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-service-ca\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.710299 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.709045 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-oauth-config\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.713683 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.713638 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-serving-cert\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.721291 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.721261 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gfld\" (UniqueName: \"kubernetes.io/projected/c215faef-0b04-4f9c-b03d-c77824db5d43-kube-api-access-7gfld\") pod \"console-7c866df8d5-llmzc\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.757366 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.757326 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:39.925683 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.925129 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85" event={"ID":"f5df1fbc-077d-4cd4-a01a-c081349e9c25","Type":"ContainerStarted","Data":"8107c738364df8e574a6e929af30f145eff6653b423d02647abf8263de8f7fae"} Jul 03 10:06:39.925683 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.925562 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85" Jul 03 10:06:39.932597 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.932538 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85" Jul 03 10:06:39.948802 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.948617 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-dsc85" podStartSLOduration=1.588796495 podStartE2EDuration="2.948596848s" totalImagesPullingTime="1.359800353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:39.946606711 +0000 UTC m=+78.012681905" watchObservedRunningTime="2026-07-03 10:06:39.948596848 +0000 UTC m=+78.014672041" Jul 03 10:06:39.975569 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:39.975435 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c866df8d5-llmzc"] Jul 03 10:06:39.983965 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:39.978415 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc215faef_0b04_4f9c_b03d_c77824db5d43.slice/crio-703f67804c6dda406c64d090bcda77524bda2cf031439a603c9f9cebdf01c25b WatchSource:0}: Error finding container 703f67804c6dda406c64d090bcda77524bda2cf031439a603c9f9cebdf01c25b: Status 404 returned error can't find the container with id 703f67804c6dda406c64d090bcda77524bda2cf031439a603c9f9cebdf01c25b Jul 03 10:06:40.438448 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.438413 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw"] Jul 03 10:06:40.488185 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.488155 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.491964 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.491937 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 03 10:06:40.492246 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.492226 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 03 10:06:40.492459 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.492441 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lbspx\"" Jul 03 10:06:40.493391 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.493363 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 03 10:06:40.495346 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.495323 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw"] Jul 03 10:06:40.616256 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.616216 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.616432 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.616269 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.616432 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.616342 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.616432 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.616386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx9hb\" (UniqueName: \"kubernetes.io/projected/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-kube-api-access-xx9hb\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.718697 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.717640 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.718697 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.717699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx9hb\" (UniqueName: \"kubernetes.io/projected/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-kube-api-access-xx9hb\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.718697 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.717757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.718697 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.717788 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.718697 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.718604 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.728576 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.728519 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.728936 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.728911 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.733081 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.733012 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx9hb\" (UniqueName: \"kubernetes.io/projected/4405e64a-4de1-4c24-b73e-a24b3e1f29e7-kube-api-access-xx9hb\") pod \"prometheus-operator-567f7b8b7c-lqcgw\" (UID: \"4405e64a-4de1-4c24-b73e-a24b3e1f29e7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.804449 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.804408 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" Jul 03 10:06:40.932646 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:40.932573 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c866df8d5-llmzc" event={"ID":"c215faef-0b04-4f9c-b03d-c77824db5d43","Type":"ContainerStarted","Data":"703f67804c6dda406c64d090bcda77524bda2cf031439a603c9f9cebdf01c25b"} Jul 03 10:06:41.488310 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:41.488027 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw"] Jul 03 10:06:41.491300 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:41.491271 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4405e64a_4de1_4c24_b73e_a24b3e1f29e7.slice/crio-eed6fb5d7a5005eab56897c9b477fd85c996cb64cd64f0727704951b42d7f1b3 WatchSource:0}: Error finding container eed6fb5d7a5005eab56897c9b477fd85c996cb64cd64f0727704951b42d7f1b3: Status 404 returned error can't find the container with id eed6fb5d7a5005eab56897c9b477fd85c996cb64cd64f0727704951b42d7f1b3 Jul 03 10:06:41.940504 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:41.940435 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" event={"ID":"4405e64a-4de1-4c24-b73e-a24b3e1f29e7","Type":"ContainerStarted","Data":"eed6fb5d7a5005eab56897c9b477fd85c996cb64cd64f0727704951b42d7f1b3"} Jul 03 10:06:45.959747 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:45.959709 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" event={"ID":"4405e64a-4de1-4c24-b73e-a24b3e1f29e7","Type":"ContainerStarted","Data":"ba1382e50dfee55ad8440fd01b136a1c49f20d5b9eb2b388ab3cc14484cebb9d"} Jul 03 10:06:45.960284 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:45.959775 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" event={"ID":"4405e64a-4de1-4c24-b73e-a24b3e1f29e7","Type":"ContainerStarted","Data":"0b85ae4ac070432d7d9d9a2b6a0df1ca3f8d3fef459d3f45b2965e87ede5452e"} Jul 03 10:06:45.961589 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:45.961559 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" event={"ID":"8ee34c3b-77d9-4c6d-9ffb-99c350fbe353","Type":"ContainerStarted","Data":"56c0bbfc212860bfc0861546dbc61647901516ebe23874c1ecb3b8ba5adbc752"} Jul 03 10:06:45.961759 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:45.961737 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:45.963461 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:45.963436 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c866df8d5-llmzc" event={"ID":"c215faef-0b04-4f9c-b03d-c77824db5d43","Type":"ContainerStarted","Data":"070aa386b7dbd2e6e12a950979aaca88ae432790a30fa62b9afcb4e1bcbf0bb1"} Jul 03 10:06:45.963694 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:45.963625 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" Jul 03 10:06:45.984476 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:45.984401 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lqcgw" podStartSLOduration=2.472165596 podStartE2EDuration="5.984380238s" totalImagesPullingTime="3.512214642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:45.980634481 +0000 UTC m=+84.046709676" watchObservedRunningTime="2026-07-03 10:06:45.984380238 +0000 UTC m=+84.050455409" Jul 03 10:06:46.003612 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:46.003557 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-869b645f9d-q94x4" podStartSLOduration=1.980519771 podStartE2EDuration="9.003541464s" totalImagesPullingTime="7.023021693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:46.002364416 +0000 UTC m=+84.068439609" watchObservedRunningTime="2026-07-03 10:06:46.003541464 +0000 UTC m=+84.069616677" Jul 03 10:06:46.026502 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:46.026442 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c866df8d5-llmzc" podStartSLOduration=2.006741583 podStartE2EDuration="7.026422585s" totalImagesPullingTime="5.019681002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:46.02475914 +0000 UTC m=+84.090834332" watchObservedRunningTime="2026-07-03 10:06:46.026422585 +0000 UTC m=+84.092497778" Jul 03 10:06:47.914198 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:47.914166 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bdd9n" Jul 03 10:06:48.105922 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.105887 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fbvn4"] Jul 03 10:06:48.108990 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.108963 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.114087 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.114062 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 03 10:06:48.114228 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.114164 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 03 10:06:48.114397 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.114379 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 03 10:06:48.114510 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.114484 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xbqqd\"" Jul 03 10:06:48.286105 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.286008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-sys\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.286105 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.286062 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-root\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.286325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.286144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-textfile\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.286325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.286178 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.286325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.286219 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqvk9\" (UniqueName: \"kubernetes.io/projected/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-kube-api-access-hqvk9\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.286325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.286249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-tls\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.286512 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.286325 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-wtmp\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.286512 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.286409 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-metrics-client-ca\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.286512 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.286452 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.387370 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.387334 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqvk9\" (UniqueName: \"kubernetes.io/projected/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-kube-api-access-hqvk9\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.387370 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.387385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-tls\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.387642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.387423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-wtmp\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.387642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.387456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-metrics-client-ca\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.387642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.387483 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.387642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.387527 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-sys\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.387642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.387574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-root\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.387642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.387603 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-textfile\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.387642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.387625 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.388397 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.388198 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-root\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.388397 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.388267 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-sys\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.388397 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.388319 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.388397 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.388359 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-wtmp\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.388697 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.388456 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-textfile\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.388697 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.388563 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-metrics-client-ca\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.390357 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.390334 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-tls\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.390460 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.390436 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.417754 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.417718 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqvk9\" (UniqueName: \"kubernetes.io/projected/eeb800ca-84ba-45c1-802a-c251a4cc7a1d-kube-api-access-hqvk9\") pod \"node-exporter-fbvn4\" (UID: \"eeb800ca-84ba-45c1-802a-c251a4cc7a1d\") " pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.420553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.420528 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fbvn4" Jul 03 10:06:48.433090 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:06:48.433057 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb800ca_84ba_45c1_802a_c251a4cc7a1d.slice/crio-39d248d3309d170a33e83bf607fb4d4a9f95bacad7539f3bf6871c5efefedef2 WatchSource:0}: Error finding container 39d248d3309d170a33e83bf607fb4d4a9f95bacad7539f3bf6871c5efefedef2: Status 404 returned error can't find the container with id 39d248d3309d170a33e83bf607fb4d4a9f95bacad7539f3bf6871c5efefedef2 Jul 03 10:06:48.981126 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:48.981062 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbvn4" event={"ID":"eeb800ca-84ba-45c1-802a-c251a4cc7a1d","Type":"ContainerStarted","Data":"39d248d3309d170a33e83bf607fb4d4a9f95bacad7539f3bf6871c5efefedef2"} Jul 03 10:06:49.757890 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:49.757836 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:49.758109 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:49.757924 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:06:49.760113 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:49.759790 2577 patch_prober.go:28] interesting pod/console-7c866df8d5-llmzc container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.22:8443/health\": dial tcp 10.132.0.22:8443: connect: connection refused" start-of-body= Jul 03 10:06:49.760113 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:49.759850 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7c866df8d5-llmzc" podUID="c215faef-0b04-4f9c-b03d-c77824db5d43" containerName="console" probeResult="failure" output="Get \"https://10.132.0.22:8443/health\": dial tcp 10.132.0.22:8443: connect: connection refused" Jul 03 10:06:50.385311 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:50.385255 2577 patch_prober.go:28] interesting pod/image-registry-5847694c56-wkv9n container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 03 10:06:50.385778 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:50.385344 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" podUID="9faa5c7c-106a-49eb-9b2d-074de55a3bcd" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 10:06:51.859909 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:51.859875 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:06:52.805926 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:52.805882 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c866df8d5-llmzc"] Jul 03 10:06:57.023152 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:57.023113 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-spvxp" event={"ID":"a54488f8-1418-4d53-ace8-1120c1c8fe30","Type":"ContainerStarted","Data":"920cd3a383ef38e27cfa4840db16dc207e44c61474b423476a1c76126e2f4f0c"} Jul 03 10:06:57.023647 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:57.023308 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-spvxp" Jul 03 10:06:57.024907 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:57.024875 2577 patch_prober.go:28] interesting pod/downloads-685bfb5f64-spvxp container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.20:8080/\": dial tcp 10.132.0.20:8080: connect: connection refused" start-of-body= Jul 03 10:06:57.025023 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:57.024931 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-spvxp" podUID="a54488f8-1418-4d53-ace8-1120c1c8fe30" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.20:8080/\": dial tcp 10.132.0.20:8080: connect: connection refused" Jul 03 10:06:57.044808 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:57.044739 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-spvxp" podStartSLOduration=1.276581636 podStartE2EDuration="20.04471754s" totalImagesPullingTime="18.768135904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:57.043434335 +0000 UTC m=+95.109509527" watchObservedRunningTime="2026-07-03 10:06:57.04471754 +0000 UTC m=+95.110792733" Jul 03 10:06:58.029784 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:58.029743 2577 generic.go:356] "Generic (PLEG): container finished" podID="eeb800ca-84ba-45c1-802a-c251a4cc7a1d" containerID="b71e08d07f4e77d0a298c2791f355ff05868af060ca9632f604b7f33e9d0c3bd" exitCode=0 Jul 03 10:06:58.030294 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:58.029832 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbvn4" event={"ID":"eeb800ca-84ba-45c1-802a-c251a4cc7a1d","Type":"ContainerDied","Data":"b71e08d07f4e77d0a298c2791f355ff05868af060ca9632f604b7f33e9d0c3bd"} Jul 03 10:06:58.032992 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:58.032966 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-spvxp" Jul 03 10:06:59.043062 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:59.043012 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbvn4" event={"ID":"eeb800ca-84ba-45c1-802a-c251a4cc7a1d","Type":"ContainerStarted","Data":"347a31be7920ccb9f210f3d1b6fb37c097498dae8f06b57b45610308615e6788"} Jul 03 10:06:59.043588 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:59.043137 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbvn4" event={"ID":"eeb800ca-84ba-45c1-802a-c251a4cc7a1d","Type":"ContainerStarted","Data":"74c17290f2f5c5ee83ef8ac793a88f3bd6c497cca152b8ec469374381c95cee0"} Jul 03 10:06:59.070496 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:06:59.070432 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fbvn4" podStartSLOduration=2.004876315 podStartE2EDuration="11.070412309s" totalImagesPullingTime="8.944199445s" totalInitContainerRuntime="121.336549ms" isStatefulPod=true podCreationTimestamp="2026-07-03 10:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:59.068610228 +0000 UTC m=+97.134685457" watchObservedRunningTime="2026-07-03 10:06:59.070412309 +0000 UTC m=+97.136487502" Jul 03 10:07:02.857042 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:02.857011 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hkgr6" Jul 03 10:07:03.982132 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:03.982067 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5847694c56-wkv9n"] Jul 03 10:07:17.830231 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:17.830189 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7c866df8d5-llmzc" podUID="c215faef-0b04-4f9c-b03d-c77824db5d43" containerName="console" containerID="cri-o://070aa386b7dbd2e6e12a950979aaca88ae432790a30fa62b9afcb4e1bcbf0bb1" gracePeriod=15 Jul 03 10:07:18.120229 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.120191 2577 generic.go:356] "Generic (PLEG): container finished" podID="c215faef-0b04-4f9c-b03d-c77824db5d43" containerID="070aa386b7dbd2e6e12a950979aaca88ae432790a30fa62b9afcb4e1bcbf0bb1" exitCode=0 Jul 03 10:07:18.120404 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.120252 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c866df8d5-llmzc" event={"ID":"c215faef-0b04-4f9c-b03d-c77824db5d43","Type":"ContainerDied","Data":"070aa386b7dbd2e6e12a950979aaca88ae432790a30fa62b9afcb4e1bcbf0bb1"} Jul 03 10:07:18.120404 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.120288 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c866df8d5-llmzc" event={"ID":"c215faef-0b04-4f9c-b03d-c77824db5d43","Type":"ContainerDied","Data":"703f67804c6dda406c64d090bcda77524bda2cf031439a603c9f9cebdf01c25b"} Jul 03 10:07:18.120404 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.120302 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="703f67804c6dda406c64d090bcda77524bda2cf031439a603c9f9cebdf01c25b" Jul 03 10:07:18.122936 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.122919 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:07:18.264214 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.264173 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-serving-cert\") pod \"c215faef-0b04-4f9c-b03d-c77824db5d43\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " Jul 03 10:07:18.264414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.264228 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gfld\" (UniqueName: \"kubernetes.io/projected/c215faef-0b04-4f9c-b03d-c77824db5d43-kube-api-access-7gfld\") pod \"c215faef-0b04-4f9c-b03d-c77824db5d43\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " Jul 03 10:07:18.264414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.264259 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-console-config\") pod \"c215faef-0b04-4f9c-b03d-c77824db5d43\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " Jul 03 10:07:18.264414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.264291 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-oauth-config\") pod \"c215faef-0b04-4f9c-b03d-c77824db5d43\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " Jul 03 10:07:18.264414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.264364 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-service-ca\") pod \"c215faef-0b04-4f9c-b03d-c77824db5d43\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " Jul 03 10:07:18.264414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.264406 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-oauth-serving-cert\") pod \"c215faef-0b04-4f9c-b03d-c77824db5d43\" (UID: \"c215faef-0b04-4f9c-b03d-c77824db5d43\") " Jul 03 10:07:18.264719 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.264678 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-console-config" (OuterVolumeSpecName: "console-config") pod "c215faef-0b04-4f9c-b03d-c77824db5d43" (UID: "c215faef-0b04-4f9c-b03d-c77824db5d43"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:07:18.264847 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.264820 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-service-ca" (OuterVolumeSpecName: "service-ca") pod "c215faef-0b04-4f9c-b03d-c77824db5d43" (UID: "c215faef-0b04-4f9c-b03d-c77824db5d43"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:07:18.264906 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.264883 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c215faef-0b04-4f9c-b03d-c77824db5d43" (UID: "c215faef-0b04-4f9c-b03d-c77824db5d43"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:07:18.266655 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.266625 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c215faef-0b04-4f9c-b03d-c77824db5d43" (UID: "c215faef-0b04-4f9c-b03d-c77824db5d43"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:07:18.266769 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.266671 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c215faef-0b04-4f9c-b03d-c77824db5d43" (UID: "c215faef-0b04-4f9c-b03d-c77824db5d43"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:07:18.266769 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.266710 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c215faef-0b04-4f9c-b03d-c77824db5d43-kube-api-access-7gfld" (OuterVolumeSpecName: "kube-api-access-7gfld") pod "c215faef-0b04-4f9c-b03d-c77824db5d43" (UID: "c215faef-0b04-4f9c-b03d-c77824db5d43"). InnerVolumeSpecName "kube-api-access-7gfld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:07:18.365644 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.365556 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:18.365644 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.365586 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-oauth-serving-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:18.365644 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.365596 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-serving-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:18.365644 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.365605 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gfld\" (UniqueName: \"kubernetes.io/projected/c215faef-0b04-4f9c-b03d-c77824db5d43-kube-api-access-7gfld\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:18.365644 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.365615 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c215faef-0b04-4f9c-b03d-c77824db5d43-console-config\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:18.365644 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:18.365624 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c215faef-0b04-4f9c-b03d-c77824db5d43-console-oauth-config\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:19.124155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:19.124123 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c866df8d5-llmzc" Jul 03 10:07:19.149610 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:19.149567 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c866df8d5-llmzc"] Jul 03 10:07:19.151548 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:19.151518 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c866df8d5-llmzc"] Jul 03 10:07:20.470606 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:20.470567 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c215faef-0b04-4f9c-b03d-c77824db5d43" path="/var/lib/kubelet/pods/c215faef-0b04-4f9c-b03d-c77824db5d43/volumes" Jul 03 10:07:29.006887 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.006826 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" podUID="9faa5c7c-106a-49eb-9b2d-074de55a3bcd" containerName="registry" containerID="cri-o://a5868b2c322a612eabb63a080a46ee20be4a7d64f33ee2cc90c44e7f20191594" gracePeriod=30 Jul 03 10:07:29.162298 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.162258 2577 generic.go:356] "Generic (PLEG): container finished" podID="9faa5c7c-106a-49eb-9b2d-074de55a3bcd" containerID="a5868b2c322a612eabb63a080a46ee20be4a7d64f33ee2cc90c44e7f20191594" exitCode=0 Jul 03 10:07:29.162475 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.162352 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" event={"ID":"9faa5c7c-106a-49eb-9b2d-074de55a3bcd","Type":"ContainerDied","Data":"a5868b2c322a612eabb63a080a46ee20be4a7d64f33ee2cc90c44e7f20191594"} Jul 03 10:07:29.299194 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.299160 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:07:29.464786 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.464744 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") pod \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " Jul 03 10:07:29.464971 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.464798 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-trusted-ca\") pod \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " Jul 03 10:07:29.464971 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.464832 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-image-registry-private-configuration\") pod \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " Jul 03 10:07:29.464971 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.464861 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rszdh\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-kube-api-access-rszdh\") pod \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " Jul 03 10:07:29.464971 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.464892 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-bound-sa-token\") pod \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " Jul 03 10:07:29.464971 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.464941 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-certificates\") pod \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " Jul 03 10:07:29.465225 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.464996 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-ca-trust-extracted\") pod \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " Jul 03 10:07:29.465225 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.465044 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-installation-pull-secrets\") pod \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\" (UID: \"9faa5c7c-106a-49eb-9b2d-074de55a3bcd\") " Jul 03 10:07:29.465655 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.465320 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9faa5c7c-106a-49eb-9b2d-074de55a3bcd" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:07:29.465655 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.465603 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9faa5c7c-106a-49eb-9b2d-074de55a3bcd" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:07:29.467615 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.467576 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9faa5c7c-106a-49eb-9b2d-074de55a3bcd" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:07:29.467751 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.467613 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-kube-api-access-rszdh" (OuterVolumeSpecName: "kube-api-access-rszdh") pod "9faa5c7c-106a-49eb-9b2d-074de55a3bcd" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd"). InnerVolumeSpecName "kube-api-access-rszdh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:07:29.467751 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.467733 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9faa5c7c-106a-49eb-9b2d-074de55a3bcd" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:07:29.467921 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.467901 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9faa5c7c-106a-49eb-9b2d-074de55a3bcd" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:07:29.467975 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.467914 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9faa5c7c-106a-49eb-9b2d-074de55a3bcd" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:07:29.474337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.474310 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9faa5c7c-106a-49eb-9b2d-074de55a3bcd" (UID: "9faa5c7c-106a-49eb-9b2d-074de55a3bcd"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:07:29.566236 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.566134 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-ca-trust-extracted\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.566236 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.566172 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-installation-pull-secrets\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.566236 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.566188 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.566236 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.566201 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-trusted-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.566236 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.566214 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-image-registry-private-configuration\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.566236 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.566227 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rszdh\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-kube-api-access-rszdh\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.566236 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.566239 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-bound-sa-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.566585 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:29.566252 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9faa5c7c-106a-49eb-9b2d-074de55a3bcd-registry-certificates\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:07:30.168437 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:30.168403 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" Jul 03 10:07:30.168877 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:30.168404 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5847694c56-wkv9n" event={"ID":"9faa5c7c-106a-49eb-9b2d-074de55a3bcd","Type":"ContainerDied","Data":"38426428e16253d56259b2dc2a5088bde6933b4be068f2f8c4ab4d803235e1f7"} Jul 03 10:07:30.168877 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:30.168537 2577 scope.go:117] "RemoveContainer" containerID="a5868b2c322a612eabb63a080a46ee20be4a7d64f33ee2cc90c44e7f20191594" Jul 03 10:07:30.170142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:30.170118 2577 generic.go:356] "Generic (PLEG): container finished" podID="41cb1751-d956-4d34-9e67-e3d994e6bf5f" containerID="aa673fa680a261b1500b91120f6898af187ee1361709ffcc32112b00bc81c625" exitCode=0 Jul 03 10:07:30.170224 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:30.170208 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" event={"ID":"41cb1751-d956-4d34-9e67-e3d994e6bf5f","Type":"ContainerDied","Data":"aa673fa680a261b1500b91120f6898af187ee1361709ffcc32112b00bc81c625"} Jul 03 10:07:30.170553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:30.170536 2577 scope.go:117] "RemoveContainer" containerID="aa673fa680a261b1500b91120f6898af187ee1361709ffcc32112b00bc81c625" Jul 03 10:07:30.209492 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:30.209451 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5847694c56-wkv9n"] Jul 03 10:07:30.212649 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:30.212627 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5847694c56-wkv9n"] Jul 03 10:07:30.471690 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:30.471610 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9faa5c7c-106a-49eb-9b2d-074de55a3bcd" path="/var/lib/kubelet/pods/9faa5c7c-106a-49eb-9b2d-074de55a3bcd/volumes" Jul 03 10:07:31.176183 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:31.176146 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqf2g" event={"ID":"41cb1751-d956-4d34-9e67-e3d994e6bf5f","Type":"ContainerStarted","Data":"6bc9e06c0f7da39471d9d28cc34e9ca70caec8b285da96b95cce51a8f4ad553e"} Jul 03 10:07:39.207734 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:39.207642 2577 generic.go:356] "Generic (PLEG): container finished" podID="0293a9da-1f5a-4a5a-8d5d-8619a7735b2f" containerID="cd4fc3131cefe19139e1e95de566fc47d0dce17c991cdb51d87c50b58c49a286" exitCode=0 Jul 03 10:07:39.208251 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:39.207723 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" event={"ID":"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f","Type":"ContainerDied","Data":"cd4fc3131cefe19139e1e95de566fc47d0dce17c991cdb51d87c50b58c49a286"} Jul 03 10:07:39.208251 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:39.208164 2577 scope.go:117] "RemoveContainer" containerID="cd4fc3131cefe19139e1e95de566fc47d0dce17c991cdb51d87c50b58c49a286" Jul 03 10:07:39.209243 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:39.209219 2577 generic.go:356] "Generic (PLEG): container finished" podID="a7e396d0-1598-4e1b-8203-660c98a64ea7" containerID="1e700b7ba9f268ea8b94a227b56518cf6101161137e262c8b3ee001e9f9e7511" exitCode=0 Jul 03 10:07:39.209352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:39.209256 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" event={"ID":"a7e396d0-1598-4e1b-8203-660c98a64ea7","Type":"ContainerDied","Data":"1e700b7ba9f268ea8b94a227b56518cf6101161137e262c8b3ee001e9f9e7511"} Jul 03 10:07:39.209509 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:39.209497 2577 scope.go:117] "RemoveContainer" containerID="1e700b7ba9f268ea8b94a227b56518cf6101161137e262c8b3ee001e9f9e7511" Jul 03 10:07:40.214337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:40.214306 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-tbrbs" event={"ID":"0293a9da-1f5a-4a5a-8d5d-8619a7735b2f","Type":"ContainerStarted","Data":"69ae4ea6ef0feb58b7b715de34cc677b37339380e5ac78eedafcb20cedc9d687"} Jul 03 10:07:40.216030 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:07:40.216009 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rhcp6" event={"ID":"a7e396d0-1598-4e1b-8203-660c98a64ea7","Type":"ContainerStarted","Data":"56bad73869823b873b76abd72b37bc13f5b13d0f913ad973cb8974ac939bea68"} Jul 03 10:08:16.175005 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.174919 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68594df7c-dmrck"] Jul 03 10:08:16.175467 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.175245 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9faa5c7c-106a-49eb-9b2d-074de55a3bcd" containerName="registry" Jul 03 10:08:16.175467 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.175259 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9faa5c7c-106a-49eb-9b2d-074de55a3bcd" containerName="registry" Jul 03 10:08:16.175467 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.175277 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c215faef-0b04-4f9c-b03d-c77824db5d43" containerName="console" Jul 03 10:08:16.175467 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.175283 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c215faef-0b04-4f9c-b03d-c77824db5d43" containerName="console" Jul 03 10:08:16.175467 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.175337 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9faa5c7c-106a-49eb-9b2d-074de55a3bcd" containerName="registry" Jul 03 10:08:16.175467 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.175346 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c215faef-0b04-4f9c-b03d-c77824db5d43" containerName="console" Jul 03 10:08:16.177457 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.177429 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.180772 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.180747 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 03 10:08:16.180890 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.180805 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 03 10:08:16.182190 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.182160 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b5rfb\"" Jul 03 10:08:16.182309 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.182218 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 03 10:08:16.182309 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.182242 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 03 10:08:16.182309 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.182247 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 03 10:08:16.186944 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.186922 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 03 10:08:16.201628 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.201592 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68594df7c-dmrck"] Jul 03 10:08:16.240887 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.240851 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-serving-cert\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.241060 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.240979 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-oauth-serving-cert\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.241060 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.241037 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-trusted-ca-bundle\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.241181 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.241155 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-service-ca\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.241304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.241229 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-oauth-config\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.241304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.241270 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msbpd\" (UniqueName: \"kubernetes.io/projected/2d6e843c-62b3-453a-931c-975be290826e-kube-api-access-msbpd\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.241304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.241298 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-console-config\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.342364 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.342320 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-service-ca\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.342555 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.342383 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-oauth-config\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.342555 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.342412 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msbpd\" (UniqueName: \"kubernetes.io/projected/2d6e843c-62b3-453a-931c-975be290826e-kube-api-access-msbpd\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.342555 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.342435 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-console-config\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.342717 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.342575 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-serving-cert\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.342717 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.342642 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-oauth-serving-cert\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.342717 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.342680 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-trusted-ca-bundle\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.343153 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.343126 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-service-ca\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.343349 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.343325 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-console-config\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.343431 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.343377 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-oauth-serving-cert\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.343620 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.343599 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-trusted-ca-bundle\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.344942 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.344919 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-oauth-config\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.345109 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.345077 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-serving-cert\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.352746 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.352720 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msbpd\" (UniqueName: \"kubernetes.io/projected/2d6e843c-62b3-453a-931c-975be290826e-kube-api-access-msbpd\") pod \"console-68594df7c-dmrck\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.487767 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.487672 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:16.641263 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:16.641230 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68594df7c-dmrck"] Jul 03 10:08:16.644791 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:08:16.644761 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6e843c_62b3_453a_931c_975be290826e.slice/crio-b8f7ccf36231295eac672772e30d14a80b789907107e04a02762fca4622cd46e WatchSource:0}: Error finding container b8f7ccf36231295eac672772e30d14a80b789907107e04a02762fca4622cd46e: Status 404 returned error can't find the container with id b8f7ccf36231295eac672772e30d14a80b789907107e04a02762fca4622cd46e Jul 03 10:08:17.365643 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:17.365605 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68594df7c-dmrck" event={"ID":"2d6e843c-62b3-453a-931c-975be290826e","Type":"ContainerStarted","Data":"857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086"} Jul 03 10:08:17.366033 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:17.365654 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68594df7c-dmrck" event={"ID":"2d6e843c-62b3-453a-931c-975be290826e","Type":"ContainerStarted","Data":"b8f7ccf36231295eac672772e30d14a80b789907107e04a02762fca4622cd46e"} Jul 03 10:08:17.385637 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:17.385575 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68594df7c-dmrck" podStartSLOduration=1.385558503 podStartE2EDuration="1.385558503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:08:17.38500794 +0000 UTC m=+175.451083133" watchObservedRunningTime="2026-07-03 10:08:17.385558503 +0000 UTC m=+175.451633694" Jul 03 10:08:26.488112 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:26.488072 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:26.488612 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:26.488149 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:26.492892 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:26.492867 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:08:27.416174 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:08:27.416144 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:09:35.848718 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:09:35.848684 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68594df7c-dmrck"] Jul 03 10:10:00.871381 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:00.871260 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-68594df7c-dmrck" podUID="2d6e843c-62b3-453a-931c-975be290826e" containerName="console" containerID="cri-o://857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086" gracePeriod=15 Jul 03 10:10:01.106180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.106157 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:10:01.173732 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.173702 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-console-config\") pod \"2d6e843c-62b3-453a-931c-975be290826e\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " Jul 03 10:10:01.173913 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.173746 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-serving-cert\") pod \"2d6e843c-62b3-453a-931c-975be290826e\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " Jul 03 10:10:01.173913 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.173762 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-service-ca\") pod \"2d6e843c-62b3-453a-931c-975be290826e\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " Jul 03 10:10:01.173913 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.173790 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-trusted-ca-bundle\") pod \"2d6e843c-62b3-453a-931c-975be290826e\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " Jul 03 10:10:01.173913 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.173819 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-oauth-config\") pod \"2d6e843c-62b3-453a-931c-975be290826e\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " Jul 03 10:10:01.173913 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.173851 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-oauth-serving-cert\") pod \"2d6e843c-62b3-453a-931c-975be290826e\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " Jul 03 10:10:01.173913 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.173884 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msbpd\" (UniqueName: \"kubernetes.io/projected/2d6e843c-62b3-453a-931c-975be290826e-kube-api-access-msbpd\") pod \"2d6e843c-62b3-453a-931c-975be290826e\" (UID: \"2d6e843c-62b3-453a-931c-975be290826e\") " Jul 03 10:10:01.174254 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.174218 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-console-config" (OuterVolumeSpecName: "console-config") pod "2d6e843c-62b3-453a-931c-975be290826e" (UID: "2d6e843c-62b3-453a-931c-975be290826e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:10:01.174311 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.174283 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2d6e843c-62b3-453a-931c-975be290826e" (UID: "2d6e843c-62b3-453a-931c-975be290826e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:10:01.174371 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.174317 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2d6e843c-62b3-453a-931c-975be290826e" (UID: "2d6e843c-62b3-453a-931c-975be290826e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:10:01.174371 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.174331 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-service-ca" (OuterVolumeSpecName: "service-ca") pod "2d6e843c-62b3-453a-931c-975be290826e" (UID: "2d6e843c-62b3-453a-931c-975be290826e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:10:01.175973 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.175939 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2d6e843c-62b3-453a-931c-975be290826e" (UID: "2d6e843c-62b3-453a-931c-975be290826e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:10:01.176074 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.175983 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2d6e843c-62b3-453a-931c-975be290826e" (UID: "2d6e843c-62b3-453a-931c-975be290826e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:10:01.176074 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.176016 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d6e843c-62b3-453a-931c-975be290826e-kube-api-access-msbpd" (OuterVolumeSpecName: "kube-api-access-msbpd") pod "2d6e843c-62b3-453a-931c-975be290826e" (UID: "2d6e843c-62b3-453a-931c-975be290826e"). InnerVolumeSpecName "kube-api-access-msbpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:10:01.275320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.275286 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-oauth-config\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:01.275320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.275315 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-oauth-serving-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:01.275320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.275324 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msbpd\" (UniqueName: \"kubernetes.io/projected/2d6e843c-62b3-453a-931c-975be290826e-kube-api-access-msbpd\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:01.275597 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.275333 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-console-config\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:01.275597 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.275341 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d6e843c-62b3-453a-931c-975be290826e-console-serving-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:01.275597 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.275349 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:01.275597 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.275357 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6e843c-62b3-453a-931c-975be290826e-trusted-ca-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:01.769961 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.769929 2577 generic.go:356] "Generic (PLEG): container finished" podID="2d6e843c-62b3-453a-931c-975be290826e" containerID="857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086" exitCode=0 Jul 03 10:10:01.770213 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.769984 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68594df7c-dmrck" Jul 03 10:10:01.770213 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.769994 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68594df7c-dmrck" event={"ID":"2d6e843c-62b3-453a-931c-975be290826e","Type":"ContainerDied","Data":"857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086"} Jul 03 10:10:01.770213 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.770036 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68594df7c-dmrck" event={"ID":"2d6e843c-62b3-453a-931c-975be290826e","Type":"ContainerDied","Data":"b8f7ccf36231295eac672772e30d14a80b789907107e04a02762fca4622cd46e"} Jul 03 10:10:01.770213 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.770057 2577 scope.go:117] "RemoveContainer" containerID="857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086" Jul 03 10:10:01.779438 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.779417 2577 scope.go:117] "RemoveContainer" containerID="857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086" Jul 03 10:10:01.779727 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:10:01.779707 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086\": container with ID starting with 857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086 not found: ID does not exist" containerID="857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086" Jul 03 10:10:01.779774 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.779736 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086"} err="failed to get container status \"857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086\": rpc error: code = NotFound desc = could not find container \"857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086\": container with ID starting with 857ac4ed1837821a34f526f7e1af29df0590a0bbd889057a631f95b1a61e9086 not found: ID does not exist" Jul 03 10:10:01.800247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.800214 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68594df7c-dmrck"] Jul 03 10:10:01.804813 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:01.804789 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68594df7c-dmrck"] Jul 03 10:10:02.470366 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:02.470332 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d6e843c-62b3-453a-931c-975be290826e" path="/var/lib/kubelet/pods/2d6e843c-62b3-453a-931c-975be290826e/volumes" Jul 03 10:10:17.939557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.939521 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5"] Jul 03 10:10:17.940020 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.939804 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d6e843c-62b3-453a-931c-975be290826e" containerName="console" Jul 03 10:10:17.940020 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.939815 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d6e843c-62b3-453a-931c-975be290826e" containerName="console" Jul 03 10:10:17.940020 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.939866 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d6e843c-62b3-453a-931c-975be290826e" containerName="console" Jul 03 10:10:17.941754 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.941734 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:17.944852 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.944827 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 03 10:10:17.946184 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.946161 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 03 10:10:17.946297 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.946228 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6dcq4\"" Jul 03 10:10:17.952484 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.952460 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5"] Jul 03 10:10:17.998001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.997957 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg4dc\" (UniqueName: \"kubernetes.io/projected/f138c747-3511-4f50-8546-1819f7035818-kube-api-access-gg4dc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:17.998200 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.998014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:17.998200 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:17.998035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:18.099476 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:18.099433 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg4dc\" (UniqueName: \"kubernetes.io/projected/f138c747-3511-4f50-8546-1819f7035818-kube-api-access-gg4dc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:18.099647 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:18.099488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:18.099647 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:18.099511 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:18.099833 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:18.099819 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:18.099875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:18.099852 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:18.109771 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:18.109736 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg4dc\" (UniqueName: \"kubernetes.io/projected/f138c747-3511-4f50-8546-1819f7035818-kube-api-access-gg4dc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:18.251238 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:18.251149 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:18.382798 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:18.382772 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5"] Jul 03 10:10:18.384835 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:10:18.384789 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf138c747_3511_4f50_8546_1819f7035818.slice/crio-e499dff8364082d00b983b2eb61e29f36e1b7a72bff08a3db297961d81b947b4 WatchSource:0}: Error finding container e499dff8364082d00b983b2eb61e29f36e1b7a72bff08a3db297961d81b947b4: Status 404 returned error can't find the container with id e499dff8364082d00b983b2eb61e29f36e1b7a72bff08a3db297961d81b947b4 Jul 03 10:10:18.836419 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:18.836382 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" event={"ID":"f138c747-3511-4f50-8546-1819f7035818","Type":"ContainerStarted","Data":"e499dff8364082d00b983b2eb61e29f36e1b7a72bff08a3db297961d81b947b4"} Jul 03 10:10:22.324198 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:22.324164 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:10:22.327551 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:22.327522 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:10:22.328213 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:22.328195 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:10:22.331878 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:22.331860 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:10:22.335541 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:22.335525 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 03 10:10:25.867196 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:25.867075 2577 generic.go:356] "Generic (PLEG): container finished" podID="f138c747-3511-4f50-8546-1819f7035818" containerID="385526a2a847eda4163ef1ed2d1ad6b6ad2bdd4834857f0a9559feadf36b6b59" exitCode=0 Jul 03 10:10:25.867196 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:25.867173 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" event={"ID":"f138c747-3511-4f50-8546-1819f7035818","Type":"ContainerDied","Data":"385526a2a847eda4163ef1ed2d1ad6b6ad2bdd4834857f0a9559feadf36b6b59"} Jul 03 10:10:25.868296 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:25.868281 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:10:29.887339 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:29.887305 2577 generic.go:356] "Generic (PLEG): container finished" podID="f138c747-3511-4f50-8546-1819f7035818" containerID="72ff9cf9136a78e781e3fcecd86fc0e1ca6d00d11bc2a333d5a6cfb05b38dd2b" exitCode=0 Jul 03 10:10:29.887734 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:29.887396 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" event={"ID":"f138c747-3511-4f50-8546-1819f7035818","Type":"ContainerDied","Data":"72ff9cf9136a78e781e3fcecd86fc0e1ca6d00d11bc2a333d5a6cfb05b38dd2b"} Jul 03 10:10:37.923428 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:37.923345 2577 generic.go:356] "Generic (PLEG): container finished" podID="f138c747-3511-4f50-8546-1819f7035818" containerID="0f04c9e1f50fed61b0e2b80508073fbfa24c2aeaf107ddbd35bde8cf6475e04e" exitCode=0 Jul 03 10:10:37.923428 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:37.923384 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" event={"ID":"f138c747-3511-4f50-8546-1819f7035818","Type":"ContainerDied","Data":"0f04c9e1f50fed61b0e2b80508073fbfa24c2aeaf107ddbd35bde8cf6475e04e"} Jul 03 10:10:39.097178 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.097148 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:39.169161 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.169119 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg4dc\" (UniqueName: \"kubernetes.io/projected/f138c747-3511-4f50-8546-1819f7035818-kube-api-access-gg4dc\") pod \"f138c747-3511-4f50-8546-1819f7035818\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " Jul 03 10:10:39.169161 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.169164 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-util\") pod \"f138c747-3511-4f50-8546-1819f7035818\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " Jul 03 10:10:39.169408 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.169195 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-bundle\") pod \"f138c747-3511-4f50-8546-1819f7035818\" (UID: \"f138c747-3511-4f50-8546-1819f7035818\") " Jul 03 10:10:39.169892 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.169867 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-bundle" (OuterVolumeSpecName: "bundle") pod "f138c747-3511-4f50-8546-1819f7035818" (UID: "f138c747-3511-4f50-8546-1819f7035818"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:10:39.171396 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.171369 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f138c747-3511-4f50-8546-1819f7035818-kube-api-access-gg4dc" (OuterVolumeSpecName: "kube-api-access-gg4dc") pod "f138c747-3511-4f50-8546-1819f7035818" (UID: "f138c747-3511-4f50-8546-1819f7035818"). InnerVolumeSpecName "kube-api-access-gg4dc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:10:39.174705 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.174677 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-util" (OuterVolumeSpecName: "util") pod "f138c747-3511-4f50-8546-1819f7035818" (UID: "f138c747-3511-4f50-8546-1819f7035818"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:10:39.270141 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.270027 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg4dc\" (UniqueName: \"kubernetes.io/projected/f138c747-3511-4f50-8546-1819f7035818-kube-api-access-gg4dc\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:39.270141 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.270058 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:39.270141 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.270067 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f138c747-3511-4f50-8546-1819f7035818-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:39.933514 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.933478 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" event={"ID":"f138c747-3511-4f50-8546-1819f7035818","Type":"ContainerDied","Data":"e499dff8364082d00b983b2eb61e29f36e1b7a72bff08a3db297961d81b947b4"} Jul 03 10:10:39.933716 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.933533 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e499dff8364082d00b983b2eb61e29f36e1b7a72bff08a3db297961d81b947b4" Jul 03 10:10:39.933716 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:39.933493 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754089sh5" Jul 03 10:10:50.378358 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.378330 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp"] Jul 03 10:10:50.378859 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.378617 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f138c747-3511-4f50-8546-1819f7035818" containerName="util" Jul 03 10:10:50.378859 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.378628 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f138c747-3511-4f50-8546-1819f7035818" containerName="util" Jul 03 10:10:50.378859 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.378635 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f138c747-3511-4f50-8546-1819f7035818" containerName="pull" Jul 03 10:10:50.378859 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.378641 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f138c747-3511-4f50-8546-1819f7035818" containerName="pull" Jul 03 10:10:50.378859 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.378659 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f138c747-3511-4f50-8546-1819f7035818" containerName="extract" Jul 03 10:10:50.378859 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.378664 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f138c747-3511-4f50-8546-1819f7035818" containerName="extract" Jul 03 10:10:50.378859 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.378712 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f138c747-3511-4f50-8546-1819f7035818" containerName="extract" Jul 03 10:10:50.380709 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.380691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.384067 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.384047 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 03 10:10:50.386345 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.386325 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 03 10:10:50.386458 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.386426 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6dcq4\"" Jul 03 10:10:50.394621 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.394594 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp"] Jul 03 10:10:50.459746 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.459708 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzn6f\" (UniqueName: \"kubernetes.io/projected/79d4b2bc-6a95-4217-9430-c6026142f3f0-kube-api-access-jzn6f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.459746 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.459752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.459953 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.459777 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.560454 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.560414 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzn6f\" (UniqueName: \"kubernetes.io/projected/79d4b2bc-6a95-4217-9430-c6026142f3f0-kube-api-access-jzn6f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.560454 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.560460 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.560718 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.560500 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.560895 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.560871 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.560959 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.560908 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.581950 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.581922 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzn6f\" (UniqueName: \"kubernetes.io/projected/79d4b2bc-6a95-4217-9430-c6026142f3f0-kube-api-access-jzn6f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.689291 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.689259 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:50.823348 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.821829 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp"] Jul 03 10:10:50.977782 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.977696 2577 generic.go:356] "Generic (PLEG): container finished" podID="79d4b2bc-6a95-4217-9430-c6026142f3f0" containerID="96ae0d3f86b4b876685d8c565680b3dff9e9da51d7e2da89103d3f9095ca74f5" exitCode=0 Jul 03 10:10:50.977782 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.977764 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" event={"ID":"79d4b2bc-6a95-4217-9430-c6026142f3f0","Type":"ContainerDied","Data":"96ae0d3f86b4b876685d8c565680b3dff9e9da51d7e2da89103d3f9095ca74f5"} Jul 03 10:10:50.978010 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:50.977811 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" event={"ID":"79d4b2bc-6a95-4217-9430-c6026142f3f0","Type":"ContainerStarted","Data":"51550d5c25c4c46a9cec8087d1c703cb5bf35b3dc4946edf4c65c9fd6e8c2aec"} Jul 03 10:10:53.992800 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:53.992767 2577 generic.go:356] "Generic (PLEG): container finished" podID="79d4b2bc-6a95-4217-9430-c6026142f3f0" containerID="ce35c209b88da9a5bae7d9a486168d5425773f981fdcc16439a95ad68f6e076f" exitCode=0 Jul 03 10:10:53.993317 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:53.992808 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" event={"ID":"79d4b2bc-6a95-4217-9430-c6026142f3f0","Type":"ContainerDied","Data":"ce35c209b88da9a5bae7d9a486168d5425773f981fdcc16439a95ad68f6e076f"} Jul 03 10:10:54.999003 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:54.998968 2577 generic.go:356] "Generic (PLEG): container finished" podID="79d4b2bc-6a95-4217-9430-c6026142f3f0" containerID="2ac40cde4d4a03f2ae754e70240c8b3e15d7a59952f73d073932224bc54862a2" exitCode=0 Jul 03 10:10:54.999465 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:54.999044 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" event={"ID":"79d4b2bc-6a95-4217-9430-c6026142f3f0","Type":"ContainerDied","Data":"2ac40cde4d4a03f2ae754e70240c8b3e15d7a59952f73d073932224bc54862a2"} Jul 03 10:10:55.207046 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.206979 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd"] Jul 03 10:10:55.209054 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.209037 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" Jul 03 10:10:55.212218 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.212184 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 03 10:10:55.212218 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.212215 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-5ddrd\"" Jul 03 10:10:55.212404 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.212272 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 03 10:10:55.220426 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.220402 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd"] Jul 03 10:10:55.296365 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.296254 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/617c70d5-d473-420e-bfa4-896d2a7aae39-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ht6vd\" (UID: \"617c70d5-d473-420e-bfa4-896d2a7aae39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" Jul 03 10:10:55.296365 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.296318 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmk77\" (UniqueName: \"kubernetes.io/projected/617c70d5-d473-420e-bfa4-896d2a7aae39-kube-api-access-cmk77\") pod \"cert-manager-cainjector-559cdd8c87-ht6vd\" (UID: \"617c70d5-d473-420e-bfa4-896d2a7aae39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" Jul 03 10:10:55.397515 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.397459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/617c70d5-d473-420e-bfa4-896d2a7aae39-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ht6vd\" (UID: \"617c70d5-d473-420e-bfa4-896d2a7aae39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" Jul 03 10:10:55.397742 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.397554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmk77\" (UniqueName: \"kubernetes.io/projected/617c70d5-d473-420e-bfa4-896d2a7aae39-kube-api-access-cmk77\") pod \"cert-manager-cainjector-559cdd8c87-ht6vd\" (UID: \"617c70d5-d473-420e-bfa4-896d2a7aae39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" Jul 03 10:10:55.409455 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.409413 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/617c70d5-d473-420e-bfa4-896d2a7aae39-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ht6vd\" (UID: \"617c70d5-d473-420e-bfa4-896d2a7aae39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" Jul 03 10:10:55.409587 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.409540 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmk77\" (UniqueName: \"kubernetes.io/projected/617c70d5-d473-420e-bfa4-896d2a7aae39-kube-api-access-cmk77\") pod \"cert-manager-cainjector-559cdd8c87-ht6vd\" (UID: \"617c70d5-d473-420e-bfa4-896d2a7aae39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" Jul 03 10:10:55.527785 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.527750 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" Jul 03 10:10:55.668580 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:55.668555 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd"] Jul 03 10:10:55.670335 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:10:55.670303 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod617c70d5_d473_420e_bfa4_896d2a7aae39.slice/crio-238f851af58e8380a9bc33d589c465ccebea5c90caf5e2fd4a15c82acb2b7525 WatchSource:0}: Error finding container 238f851af58e8380a9bc33d589c465ccebea5c90caf5e2fd4a15c82acb2b7525: Status 404 returned error can't find the container with id 238f851af58e8380a9bc33d589c465ccebea5c90caf5e2fd4a15c82acb2b7525 Jul 03 10:10:56.004385 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.004350 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" event={"ID":"617c70d5-d473-420e-bfa4-896d2a7aae39","Type":"ContainerStarted","Data":"238f851af58e8380a9bc33d589c465ccebea5c90caf5e2fd4a15c82acb2b7525"} Jul 03 10:10:56.123000 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.122974 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:56.205705 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.205674 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzn6f\" (UniqueName: \"kubernetes.io/projected/79d4b2bc-6a95-4217-9430-c6026142f3f0-kube-api-access-jzn6f\") pod \"79d4b2bc-6a95-4217-9430-c6026142f3f0\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " Jul 03 10:10:56.205876 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.205739 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-util\") pod \"79d4b2bc-6a95-4217-9430-c6026142f3f0\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " Jul 03 10:10:56.205876 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.205804 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-bundle\") pod \"79d4b2bc-6a95-4217-9430-c6026142f3f0\" (UID: \"79d4b2bc-6a95-4217-9430-c6026142f3f0\") " Jul 03 10:10:56.206223 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.206196 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-bundle" (OuterVolumeSpecName: "bundle") pod "79d4b2bc-6a95-4217-9430-c6026142f3f0" (UID: "79d4b2bc-6a95-4217-9430-c6026142f3f0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:10:56.207796 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.207770 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79d4b2bc-6a95-4217-9430-c6026142f3f0-kube-api-access-jzn6f" (OuterVolumeSpecName: "kube-api-access-jzn6f") pod "79d4b2bc-6a95-4217-9430-c6026142f3f0" (UID: "79d4b2bc-6a95-4217-9430-c6026142f3f0"). InnerVolumeSpecName "kube-api-access-jzn6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:10:56.256876 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.256767 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-util" (OuterVolumeSpecName: "util") pod "79d4b2bc-6a95-4217-9430-c6026142f3f0" (UID: "79d4b2bc-6a95-4217-9430-c6026142f3f0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:10:56.307310 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.307257 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzn6f\" (UniqueName: \"kubernetes.io/projected/79d4b2bc-6a95-4217-9430-c6026142f3f0-kube-api-access-jzn6f\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:56.307310 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.307304 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:56.307310 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:56.307314 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79d4b2bc-6a95-4217-9430-c6026142f3f0-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:10:57.010488 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:57.010449 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" event={"ID":"79d4b2bc-6a95-4217-9430-c6026142f3f0","Type":"ContainerDied","Data":"51550d5c25c4c46a9cec8087d1c703cb5bf35b3dc4946edf4c65c9fd6e8c2aec"} Jul 03 10:10:57.010488 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:57.010473 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq75fp" Jul 03 10:10:57.010488 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:10:57.010497 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51550d5c25c4c46a9cec8087d1c703cb5bf35b3dc4946edf4c65c9fd6e8c2aec" Jul 03 10:11:08.201506 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.201470 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn"] Jul 03 10:11:08.202089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.201931 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79d4b2bc-6a95-4217-9430-c6026142f3f0" containerName="util" Jul 03 10:11:08.202089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.201951 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="79d4b2bc-6a95-4217-9430-c6026142f3f0" containerName="util" Jul 03 10:11:08.202089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.201981 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79d4b2bc-6a95-4217-9430-c6026142f3f0" containerName="pull" Jul 03 10:11:08.202089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.201989 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="79d4b2bc-6a95-4217-9430-c6026142f3f0" containerName="pull" Jul 03 10:11:08.202089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.202000 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79d4b2bc-6a95-4217-9430-c6026142f3f0" containerName="extract" Jul 03 10:11:08.202089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.202009 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="79d4b2bc-6a95-4217-9430-c6026142f3f0" containerName="extract" Jul 03 10:11:08.202089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.202082 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="79d4b2bc-6a95-4217-9430-c6026142f3f0" containerName="extract" Jul 03 10:11:08.204392 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.204367 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:08.207642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.207615 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6dcq4\"" Jul 03 10:11:08.208303 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.208285 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 03 10:11:08.208964 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.208944 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 03 10:11:08.222417 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.222388 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn"] Jul 03 10:11:08.306405 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.306357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rptp2\" (UniqueName: \"kubernetes.io/projected/f97ee3dd-b62f-4894-9d3f-12cba837aae9-kube-api-access-rptp2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:08.306589 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.306421 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:08.306589 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.306468 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:08.407646 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.407607 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rptp2\" (UniqueName: \"kubernetes.io/projected/f97ee3dd-b62f-4894-9d3f-12cba837aae9-kube-api-access-rptp2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:08.407646 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.407651 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:08.407868 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.407668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:08.408031 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.408011 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:08.408115 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.408077 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:08.419946 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.419911 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rptp2\" (UniqueName: \"kubernetes.io/projected/f97ee3dd-b62f-4894-9d3f-12cba837aae9-kube-api-access-rptp2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:08.520032 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:08.519949 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:09.175683 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:09.175587 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn"] Jul 03 10:11:09.178339 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:11:09.178309 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf97ee3dd_b62f_4894_9d3f_12cba837aae9.slice/crio-914730ce1490888d3f7f7f3293135be531a42d9237a8d8b638b9412c91871bc3 WatchSource:0}: Error finding container 914730ce1490888d3f7f7f3293135be531a42d9237a8d8b638b9412c91871bc3: Status 404 returned error can't find the container with id 914730ce1490888d3f7f7f3293135be531a42d9237a8d8b638b9412c91871bc3 Jul 03 10:11:10.065305 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:10.065271 2577 generic.go:356] "Generic (PLEG): container finished" podID="f97ee3dd-b62f-4894-9d3f-12cba837aae9" containerID="1fa7e2d6dc02e1b04348ebd2af3ba1685ccf5b76563d742878eddb76c2ac77cc" exitCode=0 Jul 03 10:11:10.065796 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:10.065361 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" event={"ID":"f97ee3dd-b62f-4894-9d3f-12cba837aae9","Type":"ContainerDied","Data":"1fa7e2d6dc02e1b04348ebd2af3ba1685ccf5b76563d742878eddb76c2ac77cc"} Jul 03 10:11:10.065796 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:10.065419 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" event={"ID":"f97ee3dd-b62f-4894-9d3f-12cba837aae9","Type":"ContainerStarted","Data":"914730ce1490888d3f7f7f3293135be531a42d9237a8d8b638b9412c91871bc3"} Jul 03 10:11:10.067143 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:10.067122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" event={"ID":"617c70d5-d473-420e-bfa4-896d2a7aae39","Type":"ContainerStarted","Data":"58f048f2090514b5235c8f8d60c5224d869e2fa20be4a992de224bb0b2064e2c"} Jul 03 10:11:10.102400 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:10.102345 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ht6vd" podStartSLOduration=1.716719816 podStartE2EDuration="15.102329908s" totalImagesPullingTime="13.385610092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:11:10.101228985 +0000 UTC m=+348.167304179" watchObservedRunningTime="2026-07-03 10:11:10.102329908 +0000 UTC m=+348.168405099" Jul 03 10:11:11.073680 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:11.073601 2577 generic.go:356] "Generic (PLEG): container finished" podID="f97ee3dd-b62f-4894-9d3f-12cba837aae9" containerID="4b2ba34062a690a2aef2e1328051a8decf62848d939b930a91b2a59545027117" exitCode=0 Jul 03 10:11:11.074131 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:11.073689 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" event={"ID":"f97ee3dd-b62f-4894-9d3f-12cba837aae9","Type":"ContainerDied","Data":"4b2ba34062a690a2aef2e1328051a8decf62848d939b930a91b2a59545027117"} Jul 03 10:11:12.079595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:12.079564 2577 generic.go:356] "Generic (PLEG): container finished" podID="f97ee3dd-b62f-4894-9d3f-12cba837aae9" containerID="00c71dedbe9e33745558d7b9d5b24439844cd9afc20bf09bc77bef3a98fb9ef2" exitCode=0 Jul 03 10:11:12.079974 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:12.079650 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" event={"ID":"f97ee3dd-b62f-4894-9d3f-12cba837aae9","Type":"ContainerDied","Data":"00c71dedbe9e33745558d7b9d5b24439844cd9afc20bf09bc77bef3a98fb9ef2"} Jul 03 10:11:13.206604 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:13.206577 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:13.348848 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:13.348752 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-util\") pod \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " Jul 03 10:11:13.348848 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:13.348798 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-bundle\") pod \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " Jul 03 10:11:13.349197 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:13.348855 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rptp2\" (UniqueName: \"kubernetes.io/projected/f97ee3dd-b62f-4894-9d3f-12cba837aae9-kube-api-access-rptp2\") pod \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\" (UID: \"f97ee3dd-b62f-4894-9d3f-12cba837aae9\") " Jul 03 10:11:13.349623 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:13.349594 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-bundle" (OuterVolumeSpecName: "bundle") pod "f97ee3dd-b62f-4894-9d3f-12cba837aae9" (UID: "f97ee3dd-b62f-4894-9d3f-12cba837aae9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:11:13.350972 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:13.350950 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f97ee3dd-b62f-4894-9d3f-12cba837aae9-kube-api-access-rptp2" (OuterVolumeSpecName: "kube-api-access-rptp2") pod "f97ee3dd-b62f-4894-9d3f-12cba837aae9" (UID: "f97ee3dd-b62f-4894-9d3f-12cba837aae9"). InnerVolumeSpecName "kube-api-access-rptp2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:11:13.354086 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:13.354056 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-util" (OuterVolumeSpecName: "util") pod "f97ee3dd-b62f-4894-9d3f-12cba837aae9" (UID: "f97ee3dd-b62f-4894-9d3f-12cba837aae9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:11:13.449635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:13.449592 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rptp2\" (UniqueName: \"kubernetes.io/projected/f97ee3dd-b62f-4894-9d3f-12cba837aae9-kube-api-access-rptp2\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:11:13.449635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:13.449629 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:11:13.449635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:13.449640 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f97ee3dd-b62f-4894-9d3f-12cba837aae9-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:11:14.089962 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:14.089885 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" event={"ID":"f97ee3dd-b62f-4894-9d3f-12cba837aae9","Type":"ContainerDied","Data":"914730ce1490888d3f7f7f3293135be531a42d9237a8d8b638b9412c91871bc3"} Jul 03 10:11:14.089962 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:14.089918 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5kdtn" Jul 03 10:11:14.089962 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:14.089930 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="914730ce1490888d3f7f7f3293135be531a42d9237a8d8b638b9412c91871bc3" Jul 03 10:11:21.558876 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.558845 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9"] Jul 03 10:11:21.559377 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.559139 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f97ee3dd-b62f-4894-9d3f-12cba837aae9" containerName="util" Jul 03 10:11:21.559377 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.559151 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f97ee3dd-b62f-4894-9d3f-12cba837aae9" containerName="util" Jul 03 10:11:21.559377 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.559163 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f97ee3dd-b62f-4894-9d3f-12cba837aae9" containerName="pull" Jul 03 10:11:21.559377 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.559168 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f97ee3dd-b62f-4894-9d3f-12cba837aae9" containerName="pull" Jul 03 10:11:21.559377 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.559180 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f97ee3dd-b62f-4894-9d3f-12cba837aae9" containerName="extract" Jul 03 10:11:21.559377 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.559187 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f97ee3dd-b62f-4894-9d3f-12cba837aae9" containerName="extract" Jul 03 10:11:21.559377 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.559235 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f97ee3dd-b62f-4894-9d3f-12cba837aae9" containerName="extract" Jul 03 10:11:21.562115 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.562084 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.570854 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.570833 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hhdl4\"" Jul 03 10:11:21.571001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.570872 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 03 10:11:21.571001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.570931 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 03 10:11:21.571147 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.571133 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 03 10:11:21.571205 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.571168 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 03 10:11:21.584220 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.584190 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9"] Jul 03 10:11:21.610346 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.610308 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57c6e0fb-235a-4e66-8027-e3b3c282eb39-webhook-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.610586 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.610363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9cpk\" (UniqueName: \"kubernetes.io/projected/57c6e0fb-235a-4e66-8027-e3b3c282eb39-kube-api-access-j9cpk\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.610586 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.610505 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/57c6e0fb-235a-4e66-8027-e3b3c282eb39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.610586 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.610540 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57c6e0fb-235a-4e66-8027-e3b3c282eb39-tmp\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.610764 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.610591 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/57c6e0fb-235a-4e66-8027-e3b3c282eb39-manifests\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.711749 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.711716 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57c6e0fb-235a-4e66-8027-e3b3c282eb39-webhook-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.711943 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.711760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9cpk\" (UniqueName: \"kubernetes.io/projected/57c6e0fb-235a-4e66-8027-e3b3c282eb39-kube-api-access-j9cpk\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.711943 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.711786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/57c6e0fb-235a-4e66-8027-e3b3c282eb39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.711943 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.711807 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57c6e0fb-235a-4e66-8027-e3b3c282eb39-tmp\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.711943 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.711835 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/57c6e0fb-235a-4e66-8027-e3b3c282eb39-manifests\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.712210 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.712193 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/57c6e0fb-235a-4e66-8027-e3b3c282eb39-manifests\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.714284 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.714257 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57c6e0fb-235a-4e66-8027-e3b3c282eb39-tmp\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.714500 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.714480 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57c6e0fb-235a-4e66-8027-e3b3c282eb39-webhook-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.714541 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.714515 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/57c6e0fb-235a-4e66-8027-e3b3c282eb39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.754323 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.754286 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9cpk\" (UniqueName: \"kubernetes.io/projected/57c6e0fb-235a-4e66-8027-e3b3c282eb39-kube-api-access-j9cpk\") pod \"opendatahub-operator-controller-manager-789547c8f-gn6c9\" (UID: \"57c6e0fb-235a-4e66-8027-e3b3c282eb39\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:21.871628 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:21.871536 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:22.035951 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:22.035916 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9"] Jul 03 10:11:22.039789 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:11:22.039758 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57c6e0fb_235a_4e66_8027_e3b3c282eb39.slice/crio-485979afa1a0ea825e461d42d2a71a458279c905a7948fc932de33d90c76d75a WatchSource:0}: Error finding container 485979afa1a0ea825e461d42d2a71a458279c905a7948fc932de33d90c76d75a: Status 404 returned error can't find the container with id 485979afa1a0ea825e461d42d2a71a458279c905a7948fc932de33d90c76d75a Jul 03 10:11:22.128798 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:22.128708 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" event={"ID":"57c6e0fb-235a-4e66-8027-e3b3c282eb39","Type":"ContainerStarted","Data":"485979afa1a0ea825e461d42d2a71a458279c905a7948fc932de33d90c76d75a"} Jul 03 10:11:26.151208 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:26.151167 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" event={"ID":"57c6e0fb-235a-4e66-8027-e3b3c282eb39","Type":"ContainerStarted","Data":"3e16f38432a2690a55320574e227976dbdca88ede5d55756c3ecf12a4d01245c"} Jul 03 10:11:27.158053 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:27.158021 2577 generic.go:356] "Generic (PLEG): container finished" podID="57c6e0fb-235a-4e66-8027-e3b3c282eb39" containerID="3e16f38432a2690a55320574e227976dbdca88ede5d55756c3ecf12a4d01245c" exitCode=0 Jul 03 10:11:27.158482 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:27.158069 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" event={"ID":"57c6e0fb-235a-4e66-8027-e3b3c282eb39","Type":"ContainerDied","Data":"3e16f38432a2690a55320574e227976dbdca88ede5d55756c3ecf12a4d01245c"} Jul 03 10:11:28.163964 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:28.163926 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" event={"ID":"57c6e0fb-235a-4e66-8027-e3b3c282eb39","Type":"ContainerStarted","Data":"359acd6476324b4a56eb830ef651bbe4c06940351325bb30e22176e6cb973475"} Jul 03 10:11:28.164428 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:28.164148 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:28.190992 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:28.190941 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" podStartSLOduration=2.408540103 podStartE2EDuration="7.190925314s" totalImagesPullingTime="3.634235019s" totalInitContainerRuntime="1.148150192s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:11:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:11:28.189085082 +0000 UTC m=+366.255160341" watchObservedRunningTime="2026-07-03 10:11:28.190925314 +0000 UTC m=+366.257000566" Jul 03 10:11:39.171050 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:39.171013 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-gn6c9" Jul 03 10:11:40.934699 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:40.934664 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4"] Jul 03 10:11:40.945042 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:40.945015 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:40.949634 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:40.949607 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 03 10:11:40.951044 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:40.951021 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 03 10:11:40.951044 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:40.951044 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 03 10:11:40.951250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:40.951131 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 03 10:11:40.951250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:40.951177 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-d6g86\"" Jul 03 10:11:40.951355 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:40.951290 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:11:40.975718 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:40.975682 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4"] Jul 03 10:11:41.068631 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.068586 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-manager-config\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.068631 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.068635 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-cert\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.068844 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.068708 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8rss\" (UniqueName: \"kubernetes.io/projected/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-kube-api-access-p8rss\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.068844 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.068747 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-metrics-cert\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.169201 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.169160 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8rss\" (UniqueName: \"kubernetes.io/projected/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-kube-api-access-p8rss\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.169370 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.169212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-metrics-cert\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.169370 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.169282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-manager-config\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.169370 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.169315 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-cert\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.170157 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.170123 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-manager-config\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.171736 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.171703 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-cert\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.171835 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.171815 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-metrics-cert\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.194147 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.194063 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8rss\" (UniqueName: \"kubernetes.io/projected/bad9b19d-1e40-4fa3-a66b-e8e7f151a57d-kube-api-access-p8rss\") pod \"lws-controller-manager-5779fb96bf-rnrd4\" (UID: \"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.254728 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.254695 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:41.416409 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:41.416384 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4"] Jul 03 10:11:41.419305 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:11:41.419277 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad9b19d_1e40_4fa3_a66b_e8e7f151a57d.slice/crio-59b494b3c065687c76e613ee6b0b0c6e1b5a7943dfc146494042428a2acf7e6c WatchSource:0}: Error finding container 59b494b3c065687c76e613ee6b0b0c6e1b5a7943dfc146494042428a2acf7e6c: Status 404 returned error can't find the container with id 59b494b3c065687c76e613ee6b0b0c6e1b5a7943dfc146494042428a2acf7e6c Jul 03 10:11:42.226949 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:42.226911 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" event={"ID":"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d","Type":"ContainerStarted","Data":"59b494b3c065687c76e613ee6b0b0c6e1b5a7943dfc146494042428a2acf7e6c"} Jul 03 10:11:48.255995 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:48.255954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" event={"ID":"bad9b19d-1e40-4fa3-a66b-e8e7f151a57d","Type":"ContainerStarted","Data":"7af10a4fd844bcbf6f8a063ad0b240e81ada6f2a3118a6472557b4cbcf3db5dc"} Jul 03 10:11:48.256457 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:48.256081 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:11:48.275560 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:48.275506 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" podStartSLOduration=2.498797933 podStartE2EDuration="8.275489356s" totalImagesPullingTime="5.776691423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:11:48.273402052 +0000 UTC m=+386.339477245" watchObservedRunningTime="2026-07-03 10:11:48.275489356 +0000 UTC m=+386.341564547" Jul 03 10:11:49.844236 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.844197 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6"] Jul 03 10:11:49.847590 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.847570 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:49.851636 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.851612 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dvswk\"" Jul 03 10:11:49.852310 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.852289 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 03 10:11:49.874633 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.874601 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6"] Jul 03 10:11:49.946155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.946115 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:49.946335 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.946168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:49.946335 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.946190 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:49.946335 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.946223 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:49.946335 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.946256 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:49.946335 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.946310 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:49.946498 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.946353 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:49.946498 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.946372 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:49.946498 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:49.946393 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r4jl\" (UniqueName: \"kubernetes.io/projected/5d3c99ce-9387-408f-abf4-0ac29400b5b7-kube-api-access-5r4jl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.047808 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.047777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.047991 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.047813 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.047991 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.047840 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.047991 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.047882 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.047991 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.047915 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.047991 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.047948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.047991 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.047983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.048298 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.048004 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.048298 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.048034 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r4jl\" (UniqueName: \"kubernetes.io/projected/5d3c99ce-9387-408f-abf4-0ac29400b5b7-kube-api-access-5r4jl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.048298 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.048275 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.048529 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.048500 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.048622 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.048531 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.048622 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.048581 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.048766 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.048744 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.050453 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.050431 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.050987 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.050969 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.059938 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.059909 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d3c99ce-9387-408f-abf4-0ac29400b5b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.064625 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.064595 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r4jl\" (UniqueName: \"kubernetes.io/projected/5d3c99ce-9387-408f-abf4-0ac29400b5b7-kube-api-access-5r4jl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bdbf6\" (UID: \"5d3c99ce-9387-408f-abf4-0ac29400b5b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.158035 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.157996 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:50.343067 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.343040 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6"] Jul 03 10:11:50.345739 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:11:50.345709 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3c99ce_9387_408f_abf4_0ac29400b5b7.slice/crio-242c9532b8e0babf0c8cfd1e5832444ce6e2c92fc198f840d0f64efd6319cbde WatchSource:0}: Error finding container 242c9532b8e0babf0c8cfd1e5832444ce6e2c92fc198f840d0f64efd6319cbde: Status 404 returned error can't find the container with id 242c9532b8e0babf0c8cfd1e5832444ce6e2c92fc198f840d0f64efd6319cbde Jul 03 10:11:50.712904 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.712864 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx"] Jul 03 10:11:50.717361 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.717341 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.722193 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.722155 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 03 10:11:50.722193 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.722161 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vpp7g\"" Jul 03 10:11:50.722405 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.722192 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 03 10:11:50.722405 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.722387 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 03 10:11:50.737386 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.737356 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx"] Jul 03 10:11:50.855961 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.855919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czcrd\" (UniqueName: \"kubernetes.io/projected/d8a88800-bef0-4891-9a13-94caaa5d5ec8-kube-api-access-czcrd\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.855961 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.855963 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.856466 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.856025 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8a88800-bef0-4891-9a13-94caaa5d5ec8-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.856466 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.856077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.856466 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.856127 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d8a88800-bef0-4891-9a13-94caaa5d5ec8-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.856466 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.856190 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.856466 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.856222 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.957304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.957259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8a88800-bef0-4891-9a13-94caaa5d5ec8-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.957523 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.957315 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.957523 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.957434 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d8a88800-bef0-4891-9a13-94caaa5d5ec8-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.957523 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.957499 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.957523 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.957523 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.957755 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.957599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czcrd\" (UniqueName: \"kubernetes.io/projected/d8a88800-bef0-4891-9a13-94caaa5d5ec8-kube-api-access-czcrd\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.957755 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.957628 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.958213 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.958182 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.959663 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.959640 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8a88800-bef0-4891-9a13-94caaa5d5ec8-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.959806 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.959789 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d8a88800-bef0-4891-9a13-94caaa5d5ec8-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.960176 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.960157 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:50.960258 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:50.960194 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:51.000592 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:51.000522 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czcrd\" (UniqueName: \"kubernetes.io/projected/d8a88800-bef0-4891-9a13-94caaa5d5ec8-kube-api-access-czcrd\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:51.014739 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:51.014715 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8a88800-bef0-4891-9a13-94caaa5d5ec8-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5xtgx\" (UID: \"d8a88800-bef0-4891-9a13-94caaa5d5ec8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:51.026678 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:51.026648 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:11:51.172586 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:51.172539 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx"] Jul 03 10:11:51.177914 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:11:51.177884 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a88800_bef0_4891_9a13_94caaa5d5ec8.slice/crio-46822199ef5fd96851e11dee3be4b87f9d5ca3dfee21102c6ade758ca720b26d WatchSource:0}: Error finding container 46822199ef5fd96851e11dee3be4b87f9d5ca3dfee21102c6ade758ca720b26d: Status 404 returned error can't find the container with id 46822199ef5fd96851e11dee3be4b87f9d5ca3dfee21102c6ade758ca720b26d Jul 03 10:11:51.281914 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:51.281805 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" event={"ID":"5d3c99ce-9387-408f-abf4-0ac29400b5b7","Type":"ContainerStarted","Data":"242c9532b8e0babf0c8cfd1e5832444ce6e2c92fc198f840d0f64efd6319cbde"} Jul 03 10:11:51.283892 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:51.283811 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" event={"ID":"d8a88800-bef0-4891-9a13-94caaa5d5ec8","Type":"ContainerStarted","Data":"46822199ef5fd96851e11dee3be4b87f9d5ca3dfee21102c6ade758ca720b26d"} Jul 03 10:11:53.174239 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:53.174193 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:11:53.174520 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:53.174280 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:11:53.174520 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:53.174314 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:11:53.298968 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:53.298061 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" event={"ID":"5d3c99ce-9387-408f-abf4-0ac29400b5b7","Type":"ContainerStarted","Data":"a6938f1d1cb6386df6d1702ebdf6dc8d64a15d57bd7262727a9049fb709b9a35"} Jul 03 10:11:53.349037 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:53.348986 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" podStartSLOduration=1.5229002980000002 podStartE2EDuration="4.348967609s" totalImagesPullingTime="2.826067311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:11:53.347136001 +0000 UTC m=+391.413211193" watchObservedRunningTime="2026-07-03 10:11:53.348967609 +0000 UTC m=+391.415042801" Jul 03 10:11:54.159136 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:54.159075 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:54.163900 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:54.163870 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:54.303744 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:54.303715 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:54.304728 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:54.304708 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bdbf6" Jul 03 10:11:59.263109 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:11:59.263071 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-rnrd4" Jul 03 10:12:04.621768 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:04.621723 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:12:04.622199 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:04.621810 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:12:05.358563 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:05.358528 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" event={"ID":"d8a88800-bef0-4891-9a13-94caaa5d5ec8","Type":"ContainerStarted","Data":"b98afb5395ee2a6ea5ecc02c6966c1f95fd19a5976e2f651bac8236e8f5aa5f9"} Jul 03 10:12:05.358756 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:05.358740 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:12:05.360629 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:05.360603 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" Jul 03 10:12:05.384570 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:05.384516 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5xtgx" podStartSLOduration=1.942510846 podStartE2EDuration="15.384501267s" totalImagesPullingTime="13.441990421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:11:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:12:05.382174162 +0000 UTC m=+403.448249356" watchObservedRunningTime="2026-07-03 10:12:05.384501267 +0000 UTC m=+403.450576458" Jul 03 10:12:58.133808 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.133726 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc"] Jul 03 10:12:58.137327 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.137306 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.140340 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.140320 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 03 10:12:58.141716 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.141688 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 03 10:12:58.141842 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.141713 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-snd9l\"" Jul 03 10:12:58.146635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.146610 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc"] Jul 03 10:12:58.235785 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.235744 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdsxh\" (UniqueName: \"kubernetes.io/projected/f6453d61-fdc7-45f3-b0fb-c8f145718e38-kube-api-access-fdsxh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.236084 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.236055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.236233 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.236144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.237571 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.237550 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc"] Jul 03 10:12:58.240727 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.240710 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.249543 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.249509 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc"] Jul 03 10:12:58.337208 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.337166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.337411 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.337218 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.337411 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.337325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.337411 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.337367 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.337610 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.337419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdsxh\" (UniqueName: \"kubernetes.io/projected/f6453d61-fdc7-45f3-b0fb-c8f145718e38-kube-api-access-fdsxh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.337610 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.337462 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkwc4\" (UniqueName: \"kubernetes.io/projected/454a3546-cfbb-490a-a4b7-fa27d71d1f78-kube-api-access-xkwc4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.337610 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.337560 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.337776 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.337697 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.346331 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.346297 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdsxh\" (UniqueName: \"kubernetes.io/projected/f6453d61-fdc7-45f3-b0fb-c8f145718e38-kube-api-access-fdsxh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.438487 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.438452 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkwc4\" (UniqueName: \"kubernetes.io/projected/454a3546-cfbb-490a-a4b7-fa27d71d1f78-kube-api-access-xkwc4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.438668 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.438500 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.438668 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.438563 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.439013 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.438990 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.439050 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.439001 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.447750 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.447725 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:12:58.448252 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.448228 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkwc4\" (UniqueName: \"kubernetes.io/projected/454a3546-cfbb-490a-a4b7-fa27d71d1f78-kube-api-access-xkwc4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.550617 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.550584 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:12:58.592425 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.592234 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc"] Jul 03 10:12:58.595139 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:12:58.595089 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6453d61_fdc7_45f3_b0fb_c8f145718e38.slice/crio-b401f3e6940af510850d72e55ee56b9e22073fc484e23fd89d4f79a9e9cfd704 WatchSource:0}: Error finding container b401f3e6940af510850d72e55ee56b9e22073fc484e23fd89d4f79a9e9cfd704: Status 404 returned error can't find the container with id b401f3e6940af510850d72e55ee56b9e22073fc484e23fd89d4f79a9e9cfd704 Jul 03 10:12:58.614059 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.614018 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" event={"ID":"f6453d61-fdc7-45f3-b0fb-c8f145718e38","Type":"ContainerStarted","Data":"b401f3e6940af510850d72e55ee56b9e22073fc484e23fd89d4f79a9e9cfd704"} Jul 03 10:12:58.624503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.624472 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh"] Jul 03 10:12:58.628058 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.628033 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:58.639563 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.639534 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh"] Jul 03 10:12:58.701957 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.701923 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc"] Jul 03 10:12:58.703496 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:12:58.703465 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod454a3546_cfbb_490a_a4b7_fa27d71d1f78.slice/crio-b1a344781ba8ac422044691985b8d49fc31ad3948d3d9bfce2921e6db97b67ef WatchSource:0}: Error finding container b1a344781ba8ac422044691985b8d49fc31ad3948d3d9bfce2921e6db97b67ef: Status 404 returned error can't find the container with id b1a344781ba8ac422044691985b8d49fc31ad3948d3d9bfce2921e6db97b67ef Jul 03 10:12:58.741907 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.741872 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:58.742125 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.741972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:58.742125 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.742068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgwqg\" (UniqueName: \"kubernetes.io/projected/d0d7020c-8bff-46fe-94f8-841f94df58ff-kube-api-access-kgwqg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:58.843573 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.843538 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:58.843814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.843612 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgwqg\" (UniqueName: \"kubernetes.io/projected/d0d7020c-8bff-46fe-94f8-841f94df58ff-kube-api-access-kgwqg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:58.843814 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.843652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:58.844076 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.844056 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:58.844179 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.844155 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:58.854289 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.854264 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgwqg\" (UniqueName: \"kubernetes.io/projected/d0d7020c-8bff-46fe-94f8-841f94df58ff-kube-api-access-kgwqg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:58.942547 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:58.942508 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:12:59.029711 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.029675 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc"] Jul 03 10:12:59.034179 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.034154 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.040352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.040317 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc"] Jul 03 10:12:59.080034 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.079999 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh"] Jul 03 10:12:59.082326 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:12:59.082304 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d7020c_8bff_46fe_94f8_841f94df58ff.slice/crio-a86fabeeb302d41fed57fe63ff5e75119b28ca6efb28976ee216af1444a301ec WatchSource:0}: Error finding container a86fabeeb302d41fed57fe63ff5e75119b28ca6efb28976ee216af1444a301ec: Status 404 returned error can't find the container with id a86fabeeb302d41fed57fe63ff5e75119b28ca6efb28976ee216af1444a301ec Jul 03 10:12:59.145881 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.145853 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r76rv\" (UniqueName: \"kubernetes.io/projected/32fc9888-034a-4c33-8eaf-08d777547812-kube-api-access-r76rv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.146235 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.145931 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.146235 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.145963 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.247153 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.247023 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r76rv\" (UniqueName: \"kubernetes.io/projected/32fc9888-034a-4c33-8eaf-08d777547812-kube-api-access-r76rv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.247153 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.247133 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.247387 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.247173 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.247522 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.247497 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.247584 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.247546 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.256834 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.256809 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r76rv\" (UniqueName: \"kubernetes.io/projected/32fc9888-034a-4c33-8eaf-08d777547812-kube-api-access-r76rv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.345195 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.345144 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:12:59.485987 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.485768 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc"] Jul 03 10:12:59.488454 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:12:59.488421 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32fc9888_034a_4c33_8eaf_08d777547812.slice/crio-2dea147c4255c78360293c67c303d822278a88dfb324251dfda95bdb223511cf WatchSource:0}: Error finding container 2dea147c4255c78360293c67c303d822278a88dfb324251dfda95bdb223511cf: Status 404 returned error can't find the container with id 2dea147c4255c78360293c67c303d822278a88dfb324251dfda95bdb223511cf Jul 03 10:12:59.620389 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.620351 2577 generic.go:356] "Generic (PLEG): container finished" podID="454a3546-cfbb-490a-a4b7-fa27d71d1f78" containerID="94d885c0dae0578ee24ee24cc80c3e6db0b3f6d2d1e42de8846dc235cb503d40" exitCode=0 Jul 03 10:12:59.620585 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.620440 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" event={"ID":"454a3546-cfbb-490a-a4b7-fa27d71d1f78","Type":"ContainerDied","Data":"94d885c0dae0578ee24ee24cc80c3e6db0b3f6d2d1e42de8846dc235cb503d40"} Jul 03 10:12:59.620585 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.620541 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" event={"ID":"454a3546-cfbb-490a-a4b7-fa27d71d1f78","Type":"ContainerStarted","Data":"b1a344781ba8ac422044691985b8d49fc31ad3948d3d9bfce2921e6db97b67ef"} Jul 03 10:12:59.622310 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.622286 2577 generic.go:356] "Generic (PLEG): container finished" podID="32fc9888-034a-4c33-8eaf-08d777547812" containerID="853795292e457596e0eab71595599fa0731630c90c20d9e8a12f5189e21a9c2c" exitCode=0 Jul 03 10:12:59.622429 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.622382 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" event={"ID":"32fc9888-034a-4c33-8eaf-08d777547812","Type":"ContainerDied","Data":"853795292e457596e0eab71595599fa0731630c90c20d9e8a12f5189e21a9c2c"} Jul 03 10:12:59.622489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.622425 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" event={"ID":"32fc9888-034a-4c33-8eaf-08d777547812","Type":"ContainerStarted","Data":"2dea147c4255c78360293c67c303d822278a88dfb324251dfda95bdb223511cf"} Jul 03 10:12:59.624047 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.624021 2577 generic.go:356] "Generic (PLEG): container finished" podID="d0d7020c-8bff-46fe-94f8-841f94df58ff" containerID="a91c868a7760e26930f2502d8db7334624752fe034c5e181a33af240487b9962" exitCode=0 Jul 03 10:12:59.624181 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.624159 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" event={"ID":"d0d7020c-8bff-46fe-94f8-841f94df58ff","Type":"ContainerDied","Data":"a91c868a7760e26930f2502d8db7334624752fe034c5e181a33af240487b9962"} Jul 03 10:12:59.624255 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.624208 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" event={"ID":"d0d7020c-8bff-46fe-94f8-841f94df58ff","Type":"ContainerStarted","Data":"a86fabeeb302d41fed57fe63ff5e75119b28ca6efb28976ee216af1444a301ec"} Jul 03 10:12:59.626132 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.626088 2577 generic.go:356] "Generic (PLEG): container finished" podID="f6453d61-fdc7-45f3-b0fb-c8f145718e38" containerID="07909e6107f862b7abe72da437ca23adbeebb432cef989a8a69e0cbc6ffe5168" exitCode=0 Jul 03 10:12:59.626223 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:12:59.626143 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" event={"ID":"f6453d61-fdc7-45f3-b0fb-c8f145718e38","Type":"ContainerDied","Data":"07909e6107f862b7abe72da437ca23adbeebb432cef989a8a69e0cbc6ffe5168"} Jul 03 10:13:01.637937 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:01.637906 2577 generic.go:356] "Generic (PLEG): container finished" podID="f6453d61-fdc7-45f3-b0fb-c8f145718e38" containerID="945ecadbf57962dd873761d626b504997e08c302c689865f5c143a65a9a3cd49" exitCode=0 Jul 03 10:13:01.638438 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:01.637984 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" event={"ID":"f6453d61-fdc7-45f3-b0fb-c8f145718e38","Type":"ContainerDied","Data":"945ecadbf57962dd873761d626b504997e08c302c689865f5c143a65a9a3cd49"} Jul 03 10:13:01.639939 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:01.639915 2577 generic.go:356] "Generic (PLEG): container finished" podID="454a3546-cfbb-490a-a4b7-fa27d71d1f78" containerID="4af7f56ff7b12401ee7a90d6750e0d8783956e16d6fcf18ef37292035dbd771b" exitCode=0 Jul 03 10:13:01.640045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:01.639954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" event={"ID":"454a3546-cfbb-490a-a4b7-fa27d71d1f78","Type":"ContainerDied","Data":"4af7f56ff7b12401ee7a90d6750e0d8783956e16d6fcf18ef37292035dbd771b"} Jul 03 10:13:01.641945 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:01.641922 2577 generic.go:356] "Generic (PLEG): container finished" podID="32fc9888-034a-4c33-8eaf-08d777547812" containerID="6438d9441548e2692d708611bd2ac88541c10c01d552b7f5c8eeab735a0d5cc3" exitCode=0 Jul 03 10:13:01.642011 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:01.641995 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" event={"ID":"32fc9888-034a-4c33-8eaf-08d777547812","Type":"ContainerDied","Data":"6438d9441548e2692d708611bd2ac88541c10c01d552b7f5c8eeab735a0d5cc3"} Jul 03 10:13:01.643937 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:01.643917 2577 generic.go:356] "Generic (PLEG): container finished" podID="d0d7020c-8bff-46fe-94f8-841f94df58ff" containerID="83a10d48e117e260415271b977630c7c55c42c29b2c4428b53df9b29f0b855ea" exitCode=0 Jul 03 10:13:01.644045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:01.643955 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" event={"ID":"d0d7020c-8bff-46fe-94f8-841f94df58ff","Type":"ContainerDied","Data":"83a10d48e117e260415271b977630c7c55c42c29b2c4428b53df9b29f0b855ea"} Jul 03 10:13:02.650732 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:02.650695 2577 generic.go:356] "Generic (PLEG): container finished" podID="32fc9888-034a-4c33-8eaf-08d777547812" containerID="40a0d2cea9c310a6615b98ff4f3cfe6068699e5d929eb0f1a231e18bf7675788" exitCode=0 Jul 03 10:13:02.651213 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:02.650772 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" event={"ID":"32fc9888-034a-4c33-8eaf-08d777547812","Type":"ContainerDied","Data":"40a0d2cea9c310a6615b98ff4f3cfe6068699e5d929eb0f1a231e18bf7675788"} Jul 03 10:13:02.652995 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:02.652966 2577 generic.go:356] "Generic (PLEG): container finished" podID="d0d7020c-8bff-46fe-94f8-841f94df58ff" containerID="db883c44efa854e6aa142a7582ffa88358bf45a4de453e9aa9271df9f0d74920" exitCode=0 Jul 03 10:13:02.653167 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:02.653057 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" event={"ID":"d0d7020c-8bff-46fe-94f8-841f94df58ff","Type":"ContainerDied","Data":"db883c44efa854e6aa142a7582ffa88358bf45a4de453e9aa9271df9f0d74920"} Jul 03 10:13:02.655223 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:02.655199 2577 generic.go:356] "Generic (PLEG): container finished" podID="f6453d61-fdc7-45f3-b0fb-c8f145718e38" containerID="dad446d8ff66000c99aab7bcaa5d63aba1d1f36274fc3da89d41e7019f212479" exitCode=0 Jul 03 10:13:02.655345 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:02.655284 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" event={"ID":"f6453d61-fdc7-45f3-b0fb-c8f145718e38","Type":"ContainerDied","Data":"dad446d8ff66000c99aab7bcaa5d63aba1d1f36274fc3da89d41e7019f212479"} Jul 03 10:13:02.657270 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:02.657248 2577 generic.go:356] "Generic (PLEG): container finished" podID="454a3546-cfbb-490a-a4b7-fa27d71d1f78" containerID="b8151c13fa619cc1ec858cea4b93bf4775e9d8e6ab9c217103747804729058f5" exitCode=0 Jul 03 10:13:02.657359 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:02.657278 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" event={"ID":"454a3546-cfbb-490a-a4b7-fa27d71d1f78","Type":"ContainerDied","Data":"b8151c13fa619cc1ec858cea4b93bf4775e9d8e6ab9c217103747804729058f5"} Jul 03 10:13:03.810913 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.810871 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:13:03.845128 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.845080 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:13:03.888157 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.888128 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-bundle\") pod \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " Jul 03 10:13:03.888291 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.888177 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkwc4\" (UniqueName: \"kubernetes.io/projected/454a3546-cfbb-490a-a4b7-fa27d71d1f78-kube-api-access-xkwc4\") pod \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " Jul 03 10:13:03.888291 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.888225 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-util\") pod \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\" (UID: \"454a3546-cfbb-490a-a4b7-fa27d71d1f78\") " Jul 03 10:13:03.888822 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.888623 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-bundle" (OuterVolumeSpecName: "bundle") pod "454a3546-cfbb-490a-a4b7-fa27d71d1f78" (UID: "454a3546-cfbb-490a-a4b7-fa27d71d1f78"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:13:03.891079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.891053 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/454a3546-cfbb-490a-a4b7-fa27d71d1f78-kube-api-access-xkwc4" (OuterVolumeSpecName: "kube-api-access-xkwc4") pod "454a3546-cfbb-490a-a4b7-fa27d71d1f78" (UID: "454a3546-cfbb-490a-a4b7-fa27d71d1f78"). InnerVolumeSpecName "kube-api-access-xkwc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:13:03.894455 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.894427 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-util" (OuterVolumeSpecName: "util") pod "454a3546-cfbb-490a-a4b7-fa27d71d1f78" (UID: "454a3546-cfbb-490a-a4b7-fa27d71d1f78"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:13:03.896324 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.896304 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:13:03.898950 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.898933 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:13:03.989822 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.989722 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-util\") pod \"d0d7020c-8bff-46fe-94f8-841f94df58ff\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " Jul 03 10:13:03.989822 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.989780 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r76rv\" (UniqueName: \"kubernetes.io/projected/32fc9888-034a-4c33-8eaf-08d777547812-kube-api-access-r76rv\") pod \"32fc9888-034a-4c33-8eaf-08d777547812\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " Jul 03 10:13:03.989822 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.989807 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-util\") pod \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " Jul 03 10:13:03.989822 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.989825 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-util\") pod \"32fc9888-034a-4c33-8eaf-08d777547812\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " Jul 03 10:13:03.990200 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.989857 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-bundle\") pod \"d0d7020c-8bff-46fe-94f8-841f94df58ff\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " Jul 03 10:13:03.990200 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.989911 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-bundle\") pod \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " Jul 03 10:13:03.990200 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.989940 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-bundle\") pod \"32fc9888-034a-4c33-8eaf-08d777547812\" (UID: \"32fc9888-034a-4c33-8eaf-08d777547812\") " Jul 03 10:13:03.990200 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.989978 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdsxh\" (UniqueName: \"kubernetes.io/projected/f6453d61-fdc7-45f3-b0fb-c8f145718e38-kube-api-access-fdsxh\") pod \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\" (UID: \"f6453d61-fdc7-45f3-b0fb-c8f145718e38\") " Jul 03 10:13:03.990200 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.990007 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgwqg\" (UniqueName: \"kubernetes.io/projected/d0d7020c-8bff-46fe-94f8-841f94df58ff-kube-api-access-kgwqg\") pod \"d0d7020c-8bff-46fe-94f8-841f94df58ff\" (UID: \"d0d7020c-8bff-46fe-94f8-841f94df58ff\") " Jul 03 10:13:03.990442 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.990294 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:03.990442 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.990315 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkwc4\" (UniqueName: \"kubernetes.io/projected/454a3546-cfbb-490a-a4b7-fa27d71d1f78-kube-api-access-xkwc4\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:03.990442 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.990331 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/454a3546-cfbb-490a-a4b7-fa27d71d1f78-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:03.990640 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.990490 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-bundle" (OuterVolumeSpecName: "bundle") pod "d0d7020c-8bff-46fe-94f8-841f94df58ff" (UID: "d0d7020c-8bff-46fe-94f8-841f94df58ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:13:03.991195 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.991167 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-bundle" (OuterVolumeSpecName: "bundle") pod "f6453d61-fdc7-45f3-b0fb-c8f145718e38" (UID: "f6453d61-fdc7-45f3-b0fb-c8f145718e38"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:13:03.991305 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.991248 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-bundle" (OuterVolumeSpecName: "bundle") pod "32fc9888-034a-4c33-8eaf-08d777547812" (UID: "32fc9888-034a-4c33-8eaf-08d777547812"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:13:03.992671 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.992639 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6453d61-fdc7-45f3-b0fb-c8f145718e38-kube-api-access-fdsxh" (OuterVolumeSpecName: "kube-api-access-fdsxh") pod "f6453d61-fdc7-45f3-b0fb-c8f145718e38" (UID: "f6453d61-fdc7-45f3-b0fb-c8f145718e38"). InnerVolumeSpecName "kube-api-access-fdsxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:13:03.993018 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.992986 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0d7020c-8bff-46fe-94f8-841f94df58ff-kube-api-access-kgwqg" (OuterVolumeSpecName: "kube-api-access-kgwqg") pod "d0d7020c-8bff-46fe-94f8-841f94df58ff" (UID: "d0d7020c-8bff-46fe-94f8-841f94df58ff"). InnerVolumeSpecName "kube-api-access-kgwqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:13:03.993018 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.992995 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32fc9888-034a-4c33-8eaf-08d777547812-kube-api-access-r76rv" (OuterVolumeSpecName: "kube-api-access-r76rv") pod "32fc9888-034a-4c33-8eaf-08d777547812" (UID: "32fc9888-034a-4c33-8eaf-08d777547812"). InnerVolumeSpecName "kube-api-access-r76rv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:13:03.996122 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.996072 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-util" (OuterVolumeSpecName: "util") pod "d0d7020c-8bff-46fe-94f8-841f94df58ff" (UID: "d0d7020c-8bff-46fe-94f8-841f94df58ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:13:03.996548 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.996529 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-util" (OuterVolumeSpecName: "util") pod "f6453d61-fdc7-45f3-b0fb-c8f145718e38" (UID: "f6453d61-fdc7-45f3-b0fb-c8f145718e38"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:13:03.996796 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:03.996777 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-util" (OuterVolumeSpecName: "util") pod "32fc9888-034a-4c33-8eaf-08d777547812" (UID: "32fc9888-034a-4c33-8eaf-08d777547812"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:13:04.091054 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.091014 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:04.091054 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.091047 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:04.091054 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.091057 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:04.091054 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.091065 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32fc9888-034a-4c33-8eaf-08d777547812-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:04.091330 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.091073 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdsxh\" (UniqueName: \"kubernetes.io/projected/f6453d61-fdc7-45f3-b0fb-c8f145718e38-kube-api-access-fdsxh\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:04.091330 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.091083 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgwqg\" (UniqueName: \"kubernetes.io/projected/d0d7020c-8bff-46fe-94f8-841f94df58ff-kube-api-access-kgwqg\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:04.091330 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.091107 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0d7020c-8bff-46fe-94f8-841f94df58ff-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:04.091330 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.091116 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r76rv\" (UniqueName: \"kubernetes.io/projected/32fc9888-034a-4c33-8eaf-08d777547812-kube-api-access-r76rv\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:04.091330 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.091124 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6453d61-fdc7-45f3-b0fb-c8f145718e38-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:13:04.669153 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.669120 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" Jul 03 10:13:04.669153 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.669125 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc" event={"ID":"454a3546-cfbb-490a-a4b7-fa27d71d1f78","Type":"ContainerDied","Data":"b1a344781ba8ac422044691985b8d49fc31ad3948d3d9bfce2921e6db97b67ef"} Jul 03 10:13:04.669349 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.669174 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1a344781ba8ac422044691985b8d49fc31ad3948d3d9bfce2921e6db97b67ef" Jul 03 10:13:04.671138 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.671089 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" event={"ID":"32fc9888-034a-4c33-8eaf-08d777547812","Type":"ContainerDied","Data":"2dea147c4255c78360293c67c303d822278a88dfb324251dfda95bdb223511cf"} Jul 03 10:13:04.671138 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.671154 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2dea147c4255c78360293c67c303d822278a88dfb324251dfda95bdb223511cf" Jul 03 10:13:04.671414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.671112 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc" Jul 03 10:13:04.673527 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.673507 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" Jul 03 10:13:04.673642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.673507 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh" event={"ID":"d0d7020c-8bff-46fe-94f8-841f94df58ff","Type":"ContainerDied","Data":"a86fabeeb302d41fed57fe63ff5e75119b28ca6efb28976ee216af1444a301ec"} Jul 03 10:13:04.673707 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.673659 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a86fabeeb302d41fed57fe63ff5e75119b28ca6efb28976ee216af1444a301ec" Jul 03 10:13:04.675615 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.675592 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" event={"ID":"f6453d61-fdc7-45f3-b0fb-c8f145718e38","Type":"ContainerDied","Data":"b401f3e6940af510850d72e55ee56b9e22073fc484e23fd89d4f79a9e9cfd704"} Jul 03 10:13:04.675741 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.675632 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b401f3e6940af510850d72e55ee56b9e22073fc484e23fd89d4f79a9e9cfd704" Jul 03 10:13:04.675741 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:04.675694 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc" Jul 03 10:13:22.361254 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:13:22.361223 2577 scope.go:117] "RemoveContainer" containerID="070aa386b7dbd2e6e12a950979aaca88ae432790a30fa62b9afcb4e1bcbf0bb1" Jul 03 10:14:20.925571 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.925485 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n"] Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.925942 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6453d61-fdc7-45f3-b0fb-c8f145718e38" containerName="extract" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.925964 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6453d61-fdc7-45f3-b0fb-c8f145718e38" containerName="extract" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.925978 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6453d61-fdc7-45f3-b0fb-c8f145718e38" containerName="util" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.925989 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6453d61-fdc7-45f3-b0fb-c8f145718e38" containerName="util" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.925999 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="454a3546-cfbb-490a-a4b7-fa27d71d1f78" containerName="util" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926008 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="454a3546-cfbb-490a-a4b7-fa27d71d1f78" containerName="util" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926017 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="454a3546-cfbb-490a-a4b7-fa27d71d1f78" containerName="extract" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926025 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="454a3546-cfbb-490a-a4b7-fa27d71d1f78" containerName="extract" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926042 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="454a3546-cfbb-490a-a4b7-fa27d71d1f78" containerName="pull" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926050 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="454a3546-cfbb-490a-a4b7-fa27d71d1f78" containerName="pull" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926059 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32fc9888-034a-4c33-8eaf-08d777547812" containerName="extract" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926069 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="32fc9888-034a-4c33-8eaf-08d777547812" containerName="extract" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926117 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32fc9888-034a-4c33-8eaf-08d777547812" containerName="util" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926125 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="32fc9888-034a-4c33-8eaf-08d777547812" containerName="util" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926135 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6453d61-fdc7-45f3-b0fb-c8f145718e38" containerName="pull" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926143 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6453d61-fdc7-45f3-b0fb-c8f145718e38" containerName="pull" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926157 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0d7020c-8bff-46fe-94f8-841f94df58ff" containerName="util" Jul 03 10:14:20.926155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926165 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0d7020c-8bff-46fe-94f8-841f94df58ff" containerName="util" Jul 03 10:14:20.927001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926175 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0d7020c-8bff-46fe-94f8-841f94df58ff" containerName="pull" Jul 03 10:14:20.927001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926183 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0d7020c-8bff-46fe-94f8-841f94df58ff" containerName="pull" Jul 03 10:14:20.927001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926191 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32fc9888-034a-4c33-8eaf-08d777547812" containerName="pull" Jul 03 10:14:20.927001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926200 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="32fc9888-034a-4c33-8eaf-08d777547812" containerName="pull" Jul 03 10:14:20.927001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926209 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0d7020c-8bff-46fe-94f8-841f94df58ff" containerName="extract" Jul 03 10:14:20.927001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926218 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0d7020c-8bff-46fe-94f8-841f94df58ff" containerName="extract" Jul 03 10:14:20.927001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926313 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="32fc9888-034a-4c33-8eaf-08d777547812" containerName="extract" Jul 03 10:14:20.927001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926326 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6453d61-fdc7-45f3-b0fb-c8f145718e38" containerName="extract" Jul 03 10:14:20.927001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926337 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0d7020c-8bff-46fe-94f8-841f94df58ff" containerName="extract" Jul 03 10:14:20.927001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.926346 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="454a3546-cfbb-490a-a4b7-fa27d71d1f78" containerName="extract" Jul 03 10:14:20.929547 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.929527 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:20.932420 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.932403 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v6ff8\"" Jul 03 10:14:20.942415 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:20.942389 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n"] Jul 03 10:14:21.026891 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.026858 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.027052 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.026906 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmj9j\" (UniqueName: \"kubernetes.io/projected/d509345f-1dc1-4756-ac38-8b41b968bee6-kube-api-access-gmj9j\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.027052 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.026927 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.027052 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.026950 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.027052 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.026965 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.027052 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.026979 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.027052 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.027017 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d509345f-1dc1-4756-ac38-8b41b968bee6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.027298 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.027059 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.027298 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.027154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.128372 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128327 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.128568 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128383 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.128568 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128553 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.128698 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.128698 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128607 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d509345f-1dc1-4756-ac38-8b41b968bee6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.128698 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128636 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.128848 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128730 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.128848 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128774 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.128848 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128797 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.128848 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128826 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmj9j\" (UniqueName: \"kubernetes.io/projected/d509345f-1dc1-4756-ac38-8b41b968bee6-kube-api-access-gmj9j\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.129020 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128886 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.129020 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.128958 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.129348 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.129322 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d509345f-1dc1-4756-ac38-8b41b968bee6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.129425 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.129322 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.131077 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.131055 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.131175 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.131057 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.137878 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.137856 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmj9j\" (UniqueName: \"kubernetes.io/projected/d509345f-1dc1-4756-ac38-8b41b968bee6-kube-api-access-gmj9j\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.138059 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.138035 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d509345f-1dc1-4756-ac38-8b41b968bee6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8pq9n\" (UID: \"d509345f-1dc1-4756-ac38-8b41b968bee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.239337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.239248 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:21.383788 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.383699 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n"] Jul 03 10:14:21.386227 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:14:21.386194 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd509345f_1dc1_4756_ac38_8b41b968bee6.slice/crio-879b3209698846f15e3dbbe1ef8de29224a6c79f95be979545522ca65f897240 WatchSource:0}: Error finding container 879b3209698846f15e3dbbe1ef8de29224a6c79f95be979545522ca65f897240: Status 404 returned error can't find the container with id 879b3209698846f15e3dbbe1ef8de29224a6c79f95be979545522ca65f897240 Jul 03 10:14:21.388334 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.388301 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:14:21.388440 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.388382 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:14:21.388440 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:21.388412 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:14:22.041776 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:22.041739 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" event={"ID":"d509345f-1dc1-4756-ac38-8b41b968bee6","Type":"ContainerStarted","Data":"6fdbeb872cf053c71a4b4104776ad9e931d5e3c76d9ffcc7a55dc305fd8b271b"} Jul 03 10:14:22.042178 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:22.041803 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" event={"ID":"d509345f-1dc1-4756-ac38-8b41b968bee6","Type":"ContainerStarted","Data":"879b3209698846f15e3dbbe1ef8de29224a6c79f95be979545522ca65f897240"} Jul 03 10:14:22.069640 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:22.069591 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" podStartSLOduration=2.069574956 podStartE2EDuration="2.069574956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:14:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:14:22.06753995 +0000 UTC m=+540.133615143" watchObservedRunningTime="2026-07-03 10:14:22.069574956 +0000 UTC m=+540.135650147" Jul 03 10:14:22.240134 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:22.240085 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:22.244831 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:22.244807 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:23.046767 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:23.046734 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:14:23.047813 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:14:23.047795 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:15:03.701270 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.701231 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bf6d579cd-wvjgv"] Jul 03 10:15:03.704494 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.704475 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:03.712716 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.712694 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kxd7r\"" Jul 03 10:15:03.712837 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.712696 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 03 10:15:03.712837 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.712696 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 03 10:15:03.730367 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.730335 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bf6d579cd-wvjgv"] Jul 03 10:15:03.807748 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.807706 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab0edf5b-a61d-4ad0-8e5c-851f10589606-webhook-cert\") pod \"maas-controller-6bf6d579cd-wvjgv\" (UID: \"ab0edf5b-a61d-4ad0-8e5c-851f10589606\") " pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:03.807918 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.807835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2c4f\" (UniqueName: \"kubernetes.io/projected/ab0edf5b-a61d-4ad0-8e5c-851f10589606-kube-api-access-v2c4f\") pod \"maas-controller-6bf6d579cd-wvjgv\" (UID: \"ab0edf5b-a61d-4ad0-8e5c-851f10589606\") " pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:03.908896 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.908854 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2c4f\" (UniqueName: \"kubernetes.io/projected/ab0edf5b-a61d-4ad0-8e5c-851f10589606-kube-api-access-v2c4f\") pod \"maas-controller-6bf6d579cd-wvjgv\" (UID: \"ab0edf5b-a61d-4ad0-8e5c-851f10589606\") " pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:03.909075 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.908914 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab0edf5b-a61d-4ad0-8e5c-851f10589606-webhook-cert\") pod \"maas-controller-6bf6d579cd-wvjgv\" (UID: \"ab0edf5b-a61d-4ad0-8e5c-851f10589606\") " pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:03.911382 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.911360 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab0edf5b-a61d-4ad0-8e5c-851f10589606-webhook-cert\") pod \"maas-controller-6bf6d579cd-wvjgv\" (UID: \"ab0edf5b-a61d-4ad0-8e5c-851f10589606\") " pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:03.927491 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.927461 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b7966cdb9-9h8sv"] Jul 03 10:15:03.930592 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.930569 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2c4f\" (UniqueName: \"kubernetes.io/projected/ab0edf5b-a61d-4ad0-8e5c-851f10589606-kube-api-access-v2c4f\") pod \"maas-controller-6bf6d579cd-wvjgv\" (UID: \"ab0edf5b-a61d-4ad0-8e5c-851f10589606\") " pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:03.931073 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.931058 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:15:03.944765 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:03.944736 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7966cdb9-9h8sv"] Jul 03 10:15:04.013818 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.013721 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:04.110759 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.110704 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjkg7\" (UniqueName: \"kubernetes.io/projected/a21854c1-8ee8-4671-bf47-e5d428ef3961-kube-api-access-mjkg7\") pod \"maas-controller-5b7966cdb9-9h8sv\" (UID: \"a21854c1-8ee8-4671-bf47-e5d428ef3961\") " pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:15:04.110912 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.110838 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a21854c1-8ee8-4671-bf47-e5d428ef3961-webhook-cert\") pod \"maas-controller-5b7966cdb9-9h8sv\" (UID: \"a21854c1-8ee8-4671-bf47-e5d428ef3961\") " pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:15:04.163032 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.162354 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bf6d579cd-wvjgv"] Jul 03 10:15:04.211847 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.211806 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a21854c1-8ee8-4671-bf47-e5d428ef3961-webhook-cert\") pod \"maas-controller-5b7966cdb9-9h8sv\" (UID: \"a21854c1-8ee8-4671-bf47-e5d428ef3961\") " pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:15:04.212028 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.211901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjkg7\" (UniqueName: \"kubernetes.io/projected/a21854c1-8ee8-4671-bf47-e5d428ef3961-kube-api-access-mjkg7\") pod \"maas-controller-5b7966cdb9-9h8sv\" (UID: \"a21854c1-8ee8-4671-bf47-e5d428ef3961\") " pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:15:04.214314 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.214283 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a21854c1-8ee8-4671-bf47-e5d428ef3961-webhook-cert\") pod \"maas-controller-5b7966cdb9-9h8sv\" (UID: \"a21854c1-8ee8-4671-bf47-e5d428ef3961\") " pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:15:04.223065 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.223035 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjkg7\" (UniqueName: \"kubernetes.io/projected/a21854c1-8ee8-4671-bf47-e5d428ef3961-kube-api-access-mjkg7\") pod \"maas-controller-5b7966cdb9-9h8sv\" (UID: \"a21854c1-8ee8-4671-bf47-e5d428ef3961\") " pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:15:04.247361 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.247330 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:15:04.247585 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.247555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" event={"ID":"ab0edf5b-a61d-4ad0-8e5c-851f10589606","Type":"ContainerStarted","Data":"bf2992eae01a153e54177bc5d2bf723d26ffb6c526eeecd861b49fcaa0d69b01"} Jul 03 10:15:04.387887 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:04.387858 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7966cdb9-9h8sv"] Jul 03 10:15:04.389773 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:15:04.389739 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda21854c1_8ee8_4671_bf47_e5d428ef3961.slice/crio-2aadc098524bef634dca8ae98227fc25c352623e32e370cb461895526f172e74 WatchSource:0}: Error finding container 2aadc098524bef634dca8ae98227fc25c352623e32e370cb461895526f172e74: Status 404 returned error can't find the container with id 2aadc098524bef634dca8ae98227fc25c352623e32e370cb461895526f172e74 Jul 03 10:15:05.257746 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:05.257658 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" event={"ID":"a21854c1-8ee8-4671-bf47-e5d428ef3961","Type":"ContainerStarted","Data":"2aadc098524bef634dca8ae98227fc25c352623e32e370cb461895526f172e74"} Jul 03 10:15:07.269863 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:07.269823 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" event={"ID":"a21854c1-8ee8-4671-bf47-e5d428ef3961","Type":"ContainerStarted","Data":"35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e"} Jul 03 10:15:07.270355 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:07.269891 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:15:07.271337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:07.271305 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" event={"ID":"ab0edf5b-a61d-4ad0-8e5c-851f10589606","Type":"ContainerStarted","Data":"dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44"} Jul 03 10:15:07.271450 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:07.271374 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:07.294570 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:07.294523 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" podStartSLOduration=2.308374217 podStartE2EDuration="4.294505406s" totalImagesPullingTime="1.986131189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:15:07.291258373 +0000 UTC m=+585.357333564" watchObservedRunningTime="2026-07-03 10:15:07.294505406 +0000 UTC m=+585.360580748" Jul 03 10:15:07.327140 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:07.327067 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" podStartSLOduration=2.127441186 podStartE2EDuration="4.327052039s" totalImagesPullingTime="2.199610853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:15:07.324450407 +0000 UTC m=+585.390525621" watchObservedRunningTime="2026-07-03 10:15:07.327052039 +0000 UTC m=+585.393127286" Jul 03 10:15:18.282327 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.282296 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:15:18.282751 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.282368 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:18.349452 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.349412 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bf6d579cd-wvjgv"] Jul 03 10:15:18.349634 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.349598 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" podUID="ab0edf5b-a61d-4ad0-8e5c-851f10589606" containerName="manager" containerID="cri-o://dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44" gracePeriod=30 Jul 03 10:15:18.597197 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.597170 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:18.646649 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.646612 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab0edf5b-a61d-4ad0-8e5c-851f10589606-webhook-cert\") pod \"ab0edf5b-a61d-4ad0-8e5c-851f10589606\" (UID: \"ab0edf5b-a61d-4ad0-8e5c-851f10589606\") " Jul 03 10:15:18.646805 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.646704 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2c4f\" (UniqueName: \"kubernetes.io/projected/ab0edf5b-a61d-4ad0-8e5c-851f10589606-kube-api-access-v2c4f\") pod \"ab0edf5b-a61d-4ad0-8e5c-851f10589606\" (UID: \"ab0edf5b-a61d-4ad0-8e5c-851f10589606\") " Jul 03 10:15:18.648957 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.648928 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab0edf5b-a61d-4ad0-8e5c-851f10589606-kube-api-access-v2c4f" (OuterVolumeSpecName: "kube-api-access-v2c4f") pod "ab0edf5b-a61d-4ad0-8e5c-851f10589606" (UID: "ab0edf5b-a61d-4ad0-8e5c-851f10589606"). InnerVolumeSpecName "kube-api-access-v2c4f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:15:18.648957 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.648945 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab0edf5b-a61d-4ad0-8e5c-851f10589606-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ab0edf5b-a61d-4ad0-8e5c-851f10589606" (UID: "ab0edf5b-a61d-4ad0-8e5c-851f10589606"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:15:18.747660 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.747625 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab0edf5b-a61d-4ad0-8e5c-851f10589606-webhook-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:15:18.747660 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:18.747657 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2c4f\" (UniqueName: \"kubernetes.io/projected/ab0edf5b-a61d-4ad0-8e5c-851f10589606-kube-api-access-v2c4f\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:15:19.341744 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:19.341713 2577 generic.go:356] "Generic (PLEG): container finished" podID="ab0edf5b-a61d-4ad0-8e5c-851f10589606" containerID="dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44" exitCode=0 Jul 03 10:15:19.342210 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:19.341759 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" Jul 03 10:15:19.342210 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:19.341783 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" event={"ID":"ab0edf5b-a61d-4ad0-8e5c-851f10589606","Type":"ContainerDied","Data":"dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44"} Jul 03 10:15:19.342210 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:19.341825 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf6d579cd-wvjgv" event={"ID":"ab0edf5b-a61d-4ad0-8e5c-851f10589606","Type":"ContainerDied","Data":"bf2992eae01a153e54177bc5d2bf723d26ffb6c526eeecd861b49fcaa0d69b01"} Jul 03 10:15:19.342210 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:19.341847 2577 scope.go:117] "RemoveContainer" containerID="dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44" Jul 03 10:15:19.350949 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:19.350799 2577 scope.go:117] "RemoveContainer" containerID="dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44" Jul 03 10:15:19.351039 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:15:19.351022 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44\": container with ID starting with dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44 not found: ID does not exist" containerID="dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44" Jul 03 10:15:19.351075 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:19.351047 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44"} err="failed to get container status \"dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44\": rpc error: code = NotFound desc = could not find container \"dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44\": container with ID starting with dc8467b71b6bd7f5300c718a88d219f7362767067e05b2eb01c4cbf7eec55c44 not found: ID does not exist" Jul 03 10:15:19.364614 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:19.364585 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bf6d579cd-wvjgv"] Jul 03 10:15:19.368243 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:19.368215 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bf6d579cd-wvjgv"] Jul 03 10:15:20.471785 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:20.471743 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab0edf5b-a61d-4ad0-8e5c-851f10589606" path="/var/lib/kubelet/pods/ab0edf5b-a61d-4ad0-8e5c-851f10589606/volumes" Jul 03 10:15:22.359727 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:22.359695 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:15:22.360622 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:22.360598 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:15:22.364015 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:22.363992 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:15:22.364746 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:22.364729 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:15:41.686237 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.686180 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5cfb4c9d6c-dsjg4"] Jul 03 10:15:41.691734 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.686546 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab0edf5b-a61d-4ad0-8e5c-851f10589606" containerName="manager" Jul 03 10:15:41.691734 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.686560 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab0edf5b-a61d-4ad0-8e5c-851f10589606" containerName="manager" Jul 03 10:15:41.691734 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.686619 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab0edf5b-a61d-4ad0-8e5c-851f10589606" containerName="manager" Jul 03 10:15:41.693082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.693065 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:41.696336 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.696305 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 03 10:15:41.696336 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.696330 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-kc4sf\"" Jul 03 10:15:41.713947 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.713919 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5cfb4c9d6c-dsjg4"] Jul 03 10:15:41.840862 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.840824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94857\" (UniqueName: \"kubernetes.io/projected/9af36f4b-77ef-493a-aa78-a27337e3a40d-kube-api-access-94857\") pod \"maas-api-5cfb4c9d6c-dsjg4\" (UID: \"9af36f4b-77ef-493a-aa78-a27337e3a40d\") " pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:41.841064 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.840888 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9af36f4b-77ef-493a-aa78-a27337e3a40d-maas-api-tls\") pod \"maas-api-5cfb4c9d6c-dsjg4\" (UID: \"9af36f4b-77ef-493a-aa78-a27337e3a40d\") " pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:41.841064 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.840914 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9af36f4b-77ef-493a-aa78-a27337e3a40d-openshift-service-ca\") pod \"maas-api-5cfb4c9d6c-dsjg4\" (UID: \"9af36f4b-77ef-493a-aa78-a27337e3a40d\") " pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:41.942220 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.942134 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9af36f4b-77ef-493a-aa78-a27337e3a40d-openshift-service-ca\") pod \"maas-api-5cfb4c9d6c-dsjg4\" (UID: \"9af36f4b-77ef-493a-aa78-a27337e3a40d\") " pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:41.942220 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.942197 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94857\" (UniqueName: \"kubernetes.io/projected/9af36f4b-77ef-493a-aa78-a27337e3a40d-kube-api-access-94857\") pod \"maas-api-5cfb4c9d6c-dsjg4\" (UID: \"9af36f4b-77ef-493a-aa78-a27337e3a40d\") " pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:41.942420 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.942242 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9af36f4b-77ef-493a-aa78-a27337e3a40d-maas-api-tls\") pod \"maas-api-5cfb4c9d6c-dsjg4\" (UID: \"9af36f4b-77ef-493a-aa78-a27337e3a40d\") " pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:41.942420 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:15:41.942339 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 03 10:15:41.942420 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:15:41.942411 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9af36f4b-77ef-493a-aa78-a27337e3a40d-maas-api-tls podName:9af36f4b-77ef-493a-aa78-a27337e3a40d nodeName:}" failed. No retries permitted until 2026-07-03 10:15:42.44239066 +0000 UTC m=+620.508465838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9af36f4b-77ef-493a-aa78-a27337e3a40d-maas-api-tls") pod "maas-api-5cfb4c9d6c-dsjg4" (UID: "9af36f4b-77ef-493a-aa78-a27337e3a40d") : secret "maas-api-serving-cert" not found Jul 03 10:15:41.942816 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.942796 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9af36f4b-77ef-493a-aa78-a27337e3a40d-openshift-service-ca\") pod \"maas-api-5cfb4c9d6c-dsjg4\" (UID: \"9af36f4b-77ef-493a-aa78-a27337e3a40d\") " pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:41.954733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:41.954701 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94857\" (UniqueName: \"kubernetes.io/projected/9af36f4b-77ef-493a-aa78-a27337e3a40d-kube-api-access-94857\") pod \"maas-api-5cfb4c9d6c-dsjg4\" (UID: \"9af36f4b-77ef-493a-aa78-a27337e3a40d\") " pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:42.447108 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:42.447057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9af36f4b-77ef-493a-aa78-a27337e3a40d-maas-api-tls\") pod \"maas-api-5cfb4c9d6c-dsjg4\" (UID: \"9af36f4b-77ef-493a-aa78-a27337e3a40d\") " pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:42.449694 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:42.449669 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9af36f4b-77ef-493a-aa78-a27337e3a40d-maas-api-tls\") pod \"maas-api-5cfb4c9d6c-dsjg4\" (UID: \"9af36f4b-77ef-493a-aa78-a27337e3a40d\") " pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:42.602715 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:42.602681 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:42.749317 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:42.749284 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5cfb4c9d6c-dsjg4"] Jul 03 10:15:42.751338 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:15:42.751304 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9af36f4b_77ef_493a_aa78_a27337e3a40d.slice/crio-29afdb8d60cec950521db92daa2aa90d532231170d91b4c256445261bc35f871 WatchSource:0}: Error finding container 29afdb8d60cec950521db92daa2aa90d532231170d91b4c256445261bc35f871: Status 404 returned error can't find the container with id 29afdb8d60cec950521db92daa2aa90d532231170d91b4c256445261bc35f871 Jul 03 10:15:42.753272 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:42.753250 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:15:43.462969 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:43.462926 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" event={"ID":"9af36f4b-77ef-493a-aa78-a27337e3a40d","Type":"ContainerStarted","Data":"29afdb8d60cec950521db92daa2aa90d532231170d91b4c256445261bc35f871"} Jul 03 10:15:45.474984 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:45.474946 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" event={"ID":"9af36f4b-77ef-493a-aa78-a27337e3a40d","Type":"ContainerStarted","Data":"3f992859338fec7327835ff8f87246a078f791363aa0c82f26cfb99bb4b0686d"} Jul 03 10:15:45.475471 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:45.475089 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:45.495948 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:45.495898 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" podStartSLOduration=2.7717296940000002 podStartE2EDuration="4.495881682s" totalImagesPullingTime="1.724151988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:15:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:15:45.494148538 +0000 UTC m=+623.560223731" watchObservedRunningTime="2026-07-03 10:15:45.495881682 +0000 UTC m=+623.561956910" Jul 03 10:15:50.501861 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:50.501825 2577 generic.go:356] "Generic (PLEG): container finished" podID="d509345f-1dc1-4756-ac38-8b41b968bee6" containerID="6fdbeb872cf053c71a4b4104776ad9e931d5e3c76d9ffcc7a55dc305fd8b271b" exitCode=0 Jul 03 10:15:50.502367 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:50.501901 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" event={"ID":"d509345f-1dc1-4756-ac38-8b41b968bee6","Type":"ContainerDied","Data":"6fdbeb872cf053c71a4b4104776ad9e931d5e3c76d9ffcc7a55dc305fd8b271b"} Jul 03 10:15:50.502367 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:50.502291 2577 scope.go:117] "RemoveContainer" containerID="6fdbeb872cf053c71a4b4104776ad9e931d5e3c76d9ffcc7a55dc305fd8b271b" Jul 03 10:15:50.503274 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:50.503247 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:15:50.503344 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:50.503313 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:15:50.503344 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:50.503341 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:15:51.485737 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:51.485711 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5cfb4c9d6c-dsjg4" Jul 03 10:15:51.509622 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:51.509588 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" event={"ID":"d509345f-1dc1-4756-ac38-8b41b968bee6","Type":"ContainerStarted","Data":"199f8d03c9e831965af4ab75de51082153e46bfebbdd4d80e9512c25a332eb0d"} Jul 03 10:15:52.241901 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:52.241862 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:15:52.242300 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:52.242277 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8pq9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 03 10:15:52.242471 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:52.242447 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" podUID="d509345f-1dc1-4756-ac38-8b41b968bee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 03 10:15:53.046717 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:53.046673 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:15:53.240437 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:53.240392 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8pq9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 03 10:15:53.240650 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:53.240468 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" podUID="d509345f-1dc1-4756-ac38-8b41b968bee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 03 10:15:54.241179 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:54.240651 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8pq9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 03 10:15:54.241179 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:54.240723 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" podUID="d509345f-1dc1-4756-ac38-8b41b968bee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 03 10:15:55.239913 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:55.239869 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8pq9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 03 10:15:55.240145 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:55.239941 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" podUID="d509345f-1dc1-4756-ac38-8b41b968bee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 03 10:15:56.243783 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:56.243751 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:15:56.244746 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:15:56.244727 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8pq9n" Jul 03 10:16:04.200586 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.200548 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q"] Jul 03 10:16:04.203287 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.203266 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.207864 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.207836 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 03 10:16:04.208422 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.208405 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5mnsz\"" Jul 03 10:16:04.208550 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.208531 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 03 10:16:04.209440 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.209413 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 03 10:16:04.220175 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.220144 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q"] Jul 03 10:16:04.349796 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.349754 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a54bb6-65c8-4fc1-82f8-b19558618cae-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.350026 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.349807 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.350026 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.349877 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.350026 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.349903 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.350026 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.349935 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kk4b\" (UniqueName: \"kubernetes.io/projected/f3a54bb6-65c8-4fc1-82f8-b19558618cae-kube-api-access-9kk4b\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.350026 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.349964 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.450542 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.450510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.450542 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.450543 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.450819 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.450573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kk4b\" (UniqueName: \"kubernetes.io/projected/f3a54bb6-65c8-4fc1-82f8-b19558618cae-kube-api-access-9kk4b\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.450819 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.450612 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.450819 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.450677 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a54bb6-65c8-4fc1-82f8-b19558618cae-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.450819 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.450721 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.451216 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.451006 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.451216 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.451111 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.451216 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.451142 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.452874 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.452853 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3a54bb6-65c8-4fc1-82f8-b19558618cae-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.453166 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.453147 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a54bb6-65c8-4fc1-82f8-b19558618cae-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.459893 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.459863 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kk4b\" (UniqueName: \"kubernetes.io/projected/f3a54bb6-65c8-4fc1-82f8-b19558618cae-kube-api-access-9kk4b\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bss7q\" (UID: \"f3a54bb6-65c8-4fc1-82f8-b19558618cae\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.513198 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.513155 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:04.654479 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:04.654454 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q"] Jul 03 10:16:04.656283 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:16:04.656259 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a54bb6_65c8_4fc1_82f8_b19558618cae.slice/crio-ff31bd338a44ba6c652ef3eb62c699737b7aa61b7089c8fd76120bb64553d54c WatchSource:0}: Error finding container ff31bd338a44ba6c652ef3eb62c699737b7aa61b7089c8fd76120bb64553d54c: Status 404 returned error can't find the container with id ff31bd338a44ba6c652ef3eb62c699737b7aa61b7089c8fd76120bb64553d54c Jul 03 10:16:05.584196 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:05.584157 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" event={"ID":"f3a54bb6-65c8-4fc1-82f8-b19558618cae","Type":"ContainerStarted","Data":"ff31bd338a44ba6c652ef3eb62c699737b7aa61b7089c8fd76120bb64553d54c"} Jul 03 10:16:06.590531 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:06.590488 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" event={"ID":"f3a54bb6-65c8-4fc1-82f8-b19558618cae","Type":"ContainerStarted","Data":"90488199cc2525bb7d7441bb85b7651554afce08e274395dd4da8e49c8d4654a"} Jul 03 10:16:06.612623 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:06.612565 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" podStartSLOduration=1.611942507 podStartE2EDuration="2.61254794s" totalImagesPullingTime="1.000605433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:16:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:16:06.611123653 +0000 UTC m=+644.677198836" watchObservedRunningTime="2026-07-03 10:16:06.61254794 +0000 UTC m=+644.678623131" Jul 03 10:16:14.513473 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:14.513416 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:14.513988 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:14.513503 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:14.526532 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:14.526503 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:14.641466 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:14.641421 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bss7q" Jul 03 10:16:19.992635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:19.992592 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm"] Jul 03 10:16:19.996902 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:19.996876 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.000173 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.000150 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 03 10:16:20.006948 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.006922 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm"] Jul 03 10:16:20.090700 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.090658 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m59bq\" (UniqueName: \"kubernetes.io/projected/0b78239f-e50a-49bf-afa7-8cc1ca170def-kube-api-access-m59bq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.090875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.090707 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b78239f-e50a-49bf-afa7-8cc1ca170def-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.090875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.090735 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.090875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.090788 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.090875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.090806 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.091016 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.090899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.191925 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.191881 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.192157 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.191936 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m59bq\" (UniqueName: \"kubernetes.io/projected/0b78239f-e50a-49bf-afa7-8cc1ca170def-kube-api-access-m59bq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.192157 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.191966 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b78239f-e50a-49bf-afa7-8cc1ca170def-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.192157 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.192005 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.192333 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.192156 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.192333 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.192212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.192443 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.192361 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.192508 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.192486 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.192553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.192518 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.194365 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.194344 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b78239f-e50a-49bf-afa7-8cc1ca170def-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.194543 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.194528 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b78239f-e50a-49bf-afa7-8cc1ca170def-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.202932 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.202911 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m59bq\" (UniqueName: \"kubernetes.io/projected/0b78239f-e50a-49bf-afa7-8cc1ca170def-kube-api-access-m59bq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm\" (UID: \"0b78239f-e50a-49bf-afa7-8cc1ca170def\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.308340 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.308248 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:20.452535 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.452509 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm"] Jul 03 10:16:20.454538 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:16:20.454517 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b78239f_e50a_49bf_afa7_8cc1ca170def.slice/crio-efc53740de9846f631aa2eb95f74a6eb8e170de67f013f70bce079c308539723 WatchSource:0}: Error finding container efc53740de9846f631aa2eb95f74a6eb8e170de67f013f70bce079c308539723: Status 404 returned error can't find the container with id efc53740de9846f631aa2eb95f74a6eb8e170de67f013f70bce079c308539723 Jul 03 10:16:20.661319 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:20.661289 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" event={"ID":"0b78239f-e50a-49bf-afa7-8cc1ca170def","Type":"ContainerStarted","Data":"efc53740de9846f631aa2eb95f74a6eb8e170de67f013f70bce079c308539723"} Jul 03 10:16:21.668264 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:21.668229 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" event={"ID":"0b78239f-e50a-49bf-afa7-8cc1ca170def","Type":"ContainerStarted","Data":"aa48b8a3409b13f74c26507880ef283eb8dc3f857452a40700859d15cfda1e92"} Jul 03 10:16:21.689618 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:21.689563 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" podStartSLOduration=2.49184985 podStartE2EDuration="2.689545693s" totalImagesPullingTime="197.695843ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:16:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:16:21.687158216 +0000 UTC m=+659.753233409" watchObservedRunningTime="2026-07-03 10:16:21.689545693 +0000 UTC m=+659.755620885" Jul 03 10:16:30.309373 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:30.309327 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:30.309757 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:30.309429 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:30.321787 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:30.321757 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:16:30.728585 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:16:30.728555 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm" Jul 03 10:19:26.171943 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.171896 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b7966cdb9-9h8sv"] Jul 03 10:19:26.172565 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.172201 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" podUID="a21854c1-8ee8-4671-bf47-e5d428ef3961" containerName="manager" containerID="cri-o://35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e" gracePeriod=30 Jul 03 10:19:26.425352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.425289 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:19:26.559669 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.559635 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjkg7\" (UniqueName: \"kubernetes.io/projected/a21854c1-8ee8-4671-bf47-e5d428ef3961-kube-api-access-mjkg7\") pod \"a21854c1-8ee8-4671-bf47-e5d428ef3961\" (UID: \"a21854c1-8ee8-4671-bf47-e5d428ef3961\") " Jul 03 10:19:26.559669 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.559672 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a21854c1-8ee8-4671-bf47-e5d428ef3961-webhook-cert\") pod \"a21854c1-8ee8-4671-bf47-e5d428ef3961\" (UID: \"a21854c1-8ee8-4671-bf47-e5d428ef3961\") " Jul 03 10:19:26.561721 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.561691 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a21854c1-8ee8-4671-bf47-e5d428ef3961-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a21854c1-8ee8-4671-bf47-e5d428ef3961" (UID: "a21854c1-8ee8-4671-bf47-e5d428ef3961"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:19:26.561831 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.561816 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a21854c1-8ee8-4671-bf47-e5d428ef3961-kube-api-access-mjkg7" (OuterVolumeSpecName: "kube-api-access-mjkg7") pod "a21854c1-8ee8-4671-bf47-e5d428ef3961" (UID: "a21854c1-8ee8-4671-bf47-e5d428ef3961"). InnerVolumeSpecName "kube-api-access-mjkg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:19:26.617405 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.617370 2577 generic.go:356] "Generic (PLEG): container finished" podID="a21854c1-8ee8-4671-bf47-e5d428ef3961" containerID="35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e" exitCode=0 Jul 03 10:19:26.617597 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.617430 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" Jul 03 10:19:26.617597 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.617461 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" event={"ID":"a21854c1-8ee8-4671-bf47-e5d428ef3961","Type":"ContainerDied","Data":"35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e"} Jul 03 10:19:26.617597 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.617518 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7966cdb9-9h8sv" event={"ID":"a21854c1-8ee8-4671-bf47-e5d428ef3961","Type":"ContainerDied","Data":"2aadc098524bef634dca8ae98227fc25c352623e32e370cb461895526f172e74"} Jul 03 10:19:26.617597 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.617547 2577 scope.go:117] "RemoveContainer" containerID="35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e" Jul 03 10:19:26.627017 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.626991 2577 scope.go:117] "RemoveContainer" containerID="35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e" Jul 03 10:19:26.627297 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:19:26.627278 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e\": container with ID starting with 35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e not found: ID does not exist" containerID="35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e" Jul 03 10:19:26.627354 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.627311 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e"} err="failed to get container status \"35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e\": rpc error: code = NotFound desc = could not find container \"35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e\": container with ID starting with 35d9c03556d52951d32b6a51f4114565eb8c3f987cb2f3332fd33105b308d97e not found: ID does not exist" Jul 03 10:19:26.660554 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.660516 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjkg7\" (UniqueName: \"kubernetes.io/projected/a21854c1-8ee8-4671-bf47-e5d428ef3961-kube-api-access-mjkg7\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:19:26.660554 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.660546 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a21854c1-8ee8-4671-bf47-e5d428ef3961-webhook-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:19:26.667043 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.667013 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b7966cdb9-9h8sv"] Jul 03 10:19:26.671732 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:26.671708 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b7966cdb9-9h8sv"] Jul 03 10:19:27.535905 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.530031 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b7966cdb9-hjdqs"] Jul 03 10:19:27.535905 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.530940 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a21854c1-8ee8-4671-bf47-e5d428ef3961" containerName="manager" Jul 03 10:19:27.535905 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.530961 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a21854c1-8ee8-4671-bf47-e5d428ef3961" containerName="manager" Jul 03 10:19:27.535905 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.531245 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a21854c1-8ee8-4671-bf47-e5d428ef3961" containerName="manager" Jul 03 10:19:27.536619 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.536595 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" Jul 03 10:19:27.539838 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.539820 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 03 10:19:27.539941 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.539837 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kxd7r\"" Jul 03 10:19:27.541275 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.541257 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 03 10:19:27.565836 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.565804 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7966cdb9-hjdqs"] Jul 03 10:19:27.667068 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.667029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7wwp\" (UniqueName: \"kubernetes.io/projected/c2ab595c-3236-4b10-853f-46f3f019bb48-kube-api-access-x7wwp\") pod \"maas-controller-5b7966cdb9-hjdqs\" (UID: \"c2ab595c-3236-4b10-853f-46f3f019bb48\") " pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" Jul 03 10:19:27.667267 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.667168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2ab595c-3236-4b10-853f-46f3f019bb48-webhook-cert\") pod \"maas-controller-5b7966cdb9-hjdqs\" (UID: \"c2ab595c-3236-4b10-853f-46f3f019bb48\") " pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" Jul 03 10:19:27.767810 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.767772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2ab595c-3236-4b10-853f-46f3f019bb48-webhook-cert\") pod \"maas-controller-5b7966cdb9-hjdqs\" (UID: \"c2ab595c-3236-4b10-853f-46f3f019bb48\") " pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" Jul 03 10:19:27.767810 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.767814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7wwp\" (UniqueName: \"kubernetes.io/projected/c2ab595c-3236-4b10-853f-46f3f019bb48-kube-api-access-x7wwp\") pod \"maas-controller-5b7966cdb9-hjdqs\" (UID: \"c2ab595c-3236-4b10-853f-46f3f019bb48\") " pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" Jul 03 10:19:27.770247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.770214 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2ab595c-3236-4b10-853f-46f3f019bb48-webhook-cert\") pod \"maas-controller-5b7966cdb9-hjdqs\" (UID: \"c2ab595c-3236-4b10-853f-46f3f019bb48\") " pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" Jul 03 10:19:27.782783 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.782759 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7wwp\" (UniqueName: \"kubernetes.io/projected/c2ab595c-3236-4b10-853f-46f3f019bb48-kube-api-access-x7wwp\") pod \"maas-controller-5b7966cdb9-hjdqs\" (UID: \"c2ab595c-3236-4b10-853f-46f3f019bb48\") " pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" Jul 03 10:19:27.846325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:27.846246 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" Jul 03 10:19:28.004444 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:28.004408 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7966cdb9-hjdqs"] Jul 03 10:19:28.013759 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:19:28.013734 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2ab595c_3236_4b10_853f_46f3f019bb48.slice/crio-827ddca14abd656fc4c4536860931ae8a9b07a0454fe5b46179b2a37c50b47f8 WatchSource:0}: Error finding container 827ddca14abd656fc4c4536860931ae8a9b07a0454fe5b46179b2a37c50b47f8: Status 404 returned error can't find the container with id 827ddca14abd656fc4c4536860931ae8a9b07a0454fe5b46179b2a37c50b47f8 Jul 03 10:19:28.472082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:28.472044 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a21854c1-8ee8-4671-bf47-e5d428ef3961" path="/var/lib/kubelet/pods/a21854c1-8ee8-4671-bf47-e5d428ef3961/volumes" Jul 03 10:19:28.630251 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:28.630210 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" event={"ID":"c2ab595c-3236-4b10-853f-46f3f019bb48","Type":"ContainerStarted","Data":"d7ee6f7c6ae81887efc878068ce52f71547e71bbcd99334b85e2ed1812064bd9"} Jul 03 10:19:28.630732 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:28.630266 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" event={"ID":"c2ab595c-3236-4b10-853f-46f3f019bb48","Type":"ContainerStarted","Data":"827ddca14abd656fc4c4536860931ae8a9b07a0454fe5b46179b2a37c50b47f8"} Jul 03 10:19:28.630732 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:28.630323 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" Jul 03 10:19:28.704175 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:28.704119 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" podStartSLOduration=1.2324457930000001 podStartE2EDuration="1.704089978s" totalImagesPullingTime="471.644185ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:19:28.698555548 +0000 UTC m=+846.764630741" watchObservedRunningTime="2026-07-03 10:19:28.704089978 +0000 UTC m=+846.770165193" Jul 03 10:19:39.641261 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:19:39.641229 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b7966cdb9-hjdqs" Jul 03 10:20:22.393702 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:20:22.393608 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:20:22.394761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:20:22.394741 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:20:22.398164 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:20:22.398144 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:20:22.398910 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:20:22.398890 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:25:22.428698 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:25:22.428617 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:25:22.430739 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:25:22.430717 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:25:22.433486 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:25:22.433464 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:25:22.435273 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:25:22.435251 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:30:22.470498 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:30:22.470462 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:30:22.474074 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:30:22.474051 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:30:22.475335 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:30:22.475314 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:30:22.478857 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:30:22.478834 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:31:18.857530 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:18.857487 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4"] Jul 03 10:31:18.861214 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:18.861187 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:18.865327 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:18.865306 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-322e29fb-openshift-default-dockercfg-9hb5z\"" Jul 03 10:31:18.891954 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:18.891920 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4"] Jul 03 10:31:19.010674 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.010639 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istiod-ca-cert\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.010674 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.010684 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-podinfo\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.010917 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.010706 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-socket\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.010917 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.010746 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-data\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.010917 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.010785 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp5mr\" (UniqueName: \"kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-kube-api-access-mp5mr\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.010917 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.010866 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-token\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.011079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.010931 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-credential-socket\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.011079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.010973 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-envoy\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.011079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.011005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-certs\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112011 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.111917 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-credential-socket\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112011 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.111975 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-envoy\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112011 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112008 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-certs\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112339 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112037 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istiod-ca-cert\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112339 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112066 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-podinfo\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112339 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112090 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-socket\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112339 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-data\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112339 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112162 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp5mr\" (UniqueName: \"kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-kube-api-access-mp5mr\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112339 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112205 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-token\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112624 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112403 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-credential-socket\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112624 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112505 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-certs\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112624 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112529 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-socket\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112624 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112566 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-data\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.112946 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.112929 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istiod-ca-cert\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.114477 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.114449 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-envoy\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.114821 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.114799 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-podinfo\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.129704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.129675 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-token\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.131064 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.131039 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp5mr\" (UniqueName: \"kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-kube-api-access-mp5mr\") pod \"e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.170824 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.170792 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:19.317662 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.317635 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4"] Jul 03 10:31:19.319340 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:31:19.319311 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3069f5d_446c_459e_b5a7_9bfdb70053bf.slice/crio-e039c0a282e53dc865c73ee2d07e7d6b0da68f3f4a0cc2dd9001f4df31e8f705 WatchSource:0}: Error finding container e039c0a282e53dc865c73ee2d07e7d6b0da68f3f4a0cc2dd9001f4df31e8f705: Status 404 returned error can't find the container with id e039c0a282e53dc865c73ee2d07e7d6b0da68f3f4a0cc2dd9001f4df31e8f705 Jul 03 10:31:19.321386 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.321353 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:31:19.321452 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.321418 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:31:19.321452 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:19.321446 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:31:20.171320 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.171282 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl"] Jul 03 10:31:20.175128 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.175084 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.178353 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.178328 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-322e29fb-serving-cert\"" Jul 03 10:31:20.187877 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.187850 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl"] Jul 03 10:31:20.304622 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.304571 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" event={"ID":"e3069f5d-446c-459e-b5a7-9bfdb70053bf","Type":"ContainerStarted","Data":"a3411fdb6fb045dd740bc206d03316890fe8436b1cbe028bb72e2c89b8ccd640"} Jul 03 10:31:20.304837 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.304644 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" event={"ID":"e3069f5d-446c-459e-b5a7-9bfdb70053bf","Type":"ContainerStarted","Data":"e039c0a282e53dc865c73ee2d07e7d6b0da68f3f4a0cc2dd9001f4df31e8f705"} Jul 03 10:31:20.322794 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.322762 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bfdf78-3341-43b8-88e2-04e64d2412e7-maas-api-tls\") pod \"maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.322989 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.322802 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7bfdf78-3341-43b8-88e2-04e64d2412e7-openshift-service-ca\") pod \"maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.322989 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.322829 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxzgz\" (UniqueName: \"kubernetes.io/projected/d7bfdf78-3341-43b8-88e2-04e64d2412e7-kube-api-access-mxzgz\") pod \"maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.342308 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.342253 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" podStartSLOduration=2.342232295 podStartE2EDuration="2.342232295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:31:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:31:20.340258415 +0000 UTC m=+1558.406333609" watchObservedRunningTime="2026-07-03 10:31:20.342232295 +0000 UTC m=+1558.408307487" Jul 03 10:31:20.424505 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.424397 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bfdf78-3341-43b8-88e2-04e64d2412e7-maas-api-tls\") pod \"maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.424505 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.424460 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7bfdf78-3341-43b8-88e2-04e64d2412e7-openshift-service-ca\") pod \"maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.424757 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.424508 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxzgz\" (UniqueName: \"kubernetes.io/projected/d7bfdf78-3341-43b8-88e2-04e64d2412e7-kube-api-access-mxzgz\") pod \"maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.425234 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.425206 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7bfdf78-3341-43b8-88e2-04e64d2412e7-openshift-service-ca\") pod \"maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.426907 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.426889 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bfdf78-3341-43b8-88e2-04e64d2412e7-maas-api-tls\") pod \"maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.440849 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.440818 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxzgz\" (UniqueName: \"kubernetes.io/projected/d7bfdf78-3341-43b8-88e2-04e64d2412e7-kube-api-access-mxzgz\") pod \"maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.488557 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.488522 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:20.633184 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.633161 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl"] Jul 03 10:31:20.635277 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:31:20.635255 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7bfdf78_3341_43b8_88e2_04e64d2412e7.slice/crio-0f415af4cfdeb8029d78abcc03d8c2f908f971c9a6f594b7c18b52e42ed57fb2 WatchSource:0}: Error finding container 0f415af4cfdeb8029d78abcc03d8c2f908f971c9a6f594b7c18b52e42ed57fb2: Status 404 returned error can't find the container with id 0f415af4cfdeb8029d78abcc03d8c2f908f971c9a6f594b7c18b52e42ed57fb2 Jul 03 10:31:20.636688 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:20.636668 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:31:21.171183 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:21.171143 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:21.175754 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:21.175726 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:21.311382 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:21.311350 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" event={"ID":"d7bfdf78-3341-43b8-88e2-04e64d2412e7","Type":"ContainerStarted","Data":"0f415af4cfdeb8029d78abcc03d8c2f908f971c9a6f594b7c18b52e42ed57fb2"} Jul 03 10:31:21.311610 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:21.311591 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:21.312517 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:21.312497 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:22.318169 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:22.318135 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" event={"ID":"d7bfdf78-3341-43b8-88e2-04e64d2412e7","Type":"ContainerStarted","Data":"910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011"} Jul 03 10:31:22.318577 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:22.318196 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:22.337986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:22.337922 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" podStartSLOduration=1.515530396 podStartE2EDuration="2.337901703s" totalImagesPullingTime="822.371307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:31:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:31:22.337408041 +0000 UTC m=+1560.403483233" watchObservedRunningTime="2026-07-03 10:31:22.337901703 +0000 UTC m=+1560.403976896" Jul 03 10:31:25.210293 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.210251 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4"] Jul 03 10:31:25.210735 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.210460 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" podUID="e3069f5d-446c-459e-b5a7-9bfdb70053bf" containerName="istio-proxy" containerID="cri-o://a3411fdb6fb045dd740bc206d03316890fe8436b1cbe028bb72e2c89b8ccd640" gracePeriod=30 Jul 03 10:31:25.463824 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.462939 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl"] Jul 03 10:31:25.463824 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.463188 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" podUID="d7bfdf78-3341-43b8-88e2-04e64d2412e7" containerName="maas-api" containerID="cri-o://910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011" gracePeriod=30 Jul 03 10:31:25.755307 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.755268 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:25.878701 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.878663 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bfdf78-3341-43b8-88e2-04e64d2412e7-maas-api-tls\") pod \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " Jul 03 10:31:25.878864 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.878760 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxzgz\" (UniqueName: \"kubernetes.io/projected/d7bfdf78-3341-43b8-88e2-04e64d2412e7-kube-api-access-mxzgz\") pod \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " Jul 03 10:31:25.878864 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.878787 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7bfdf78-3341-43b8-88e2-04e64d2412e7-openshift-service-ca\") pod \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\" (UID: \"d7bfdf78-3341-43b8-88e2-04e64d2412e7\") " Jul 03 10:31:25.879194 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.879166 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7bfdf78-3341-43b8-88e2-04e64d2412e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7bfdf78-3341-43b8-88e2-04e64d2412e7" (UID: "d7bfdf78-3341-43b8-88e2-04e64d2412e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:31:25.880754 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.880730 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7bfdf78-3341-43b8-88e2-04e64d2412e7-kube-api-access-mxzgz" (OuterVolumeSpecName: "kube-api-access-mxzgz") pod "d7bfdf78-3341-43b8-88e2-04e64d2412e7" (UID: "d7bfdf78-3341-43b8-88e2-04e64d2412e7"). InnerVolumeSpecName "kube-api-access-mxzgz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:31:25.880819 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.880752 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bfdf78-3341-43b8-88e2-04e64d2412e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7bfdf78-3341-43b8-88e2-04e64d2412e7" (UID: "d7bfdf78-3341-43b8-88e2-04e64d2412e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:31:25.979742 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.979658 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxzgz\" (UniqueName: \"kubernetes.io/projected/d7bfdf78-3341-43b8-88e2-04e64d2412e7-kube-api-access-mxzgz\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:25.979742 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.979686 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7bfdf78-3341-43b8-88e2-04e64d2412e7-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:25.979742 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:25.979696 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bfdf78-3341-43b8-88e2-04e64d2412e7-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:26.347124 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:26.347010 2577 generic.go:356] "Generic (PLEG): container finished" podID="d7bfdf78-3341-43b8-88e2-04e64d2412e7" containerID="910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011" exitCode=0 Jul 03 10:31:26.347124 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:26.347086 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" event={"ID":"d7bfdf78-3341-43b8-88e2-04e64d2412e7","Type":"ContainerDied","Data":"910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011"} Jul 03 10:31:26.347669 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:26.347162 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" event={"ID":"d7bfdf78-3341-43b8-88e2-04e64d2412e7","Type":"ContainerDied","Data":"0f415af4cfdeb8029d78abcc03d8c2f908f971c9a6f594b7c18b52e42ed57fb2"} Jul 03 10:31:26.347669 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:26.347190 2577 scope.go:117] "RemoveContainer" containerID="910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011" Jul 03 10:31:26.347852 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:26.347830 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl" Jul 03 10:31:26.361990 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:26.361866 2577 scope.go:117] "RemoveContainer" containerID="910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011" Jul 03 10:31:26.362466 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:31:26.362443 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011\": container with ID starting with 910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011 not found: ID does not exist" containerID="910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011" Jul 03 10:31:26.362553 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:26.362473 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011"} err="failed to get container status \"910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011\": rpc error: code = NotFound desc = could not find container \"910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011\": container with ID starting with 910899d6a5109cf5130ebb6b7c254f81fc40172fdcd03e6796ed179e95b2a011 not found: ID does not exist" Jul 03 10:31:26.377347 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:26.377319 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl"] Jul 03 10:31:26.382489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:26.382463 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-322e29fb-6d6d6858fc-sx6bl"] Jul 03 10:31:26.472253 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:26.472220 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7bfdf78-3341-43b8-88e2-04e64d2412e7" path="/var/lib/kubelet/pods/d7bfdf78-3341-43b8-88e2-04e64d2412e7/volumes" Jul 03 10:31:30.378826 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.378791 2577 generic.go:356] "Generic (PLEG): container finished" podID="e3069f5d-446c-459e-b5a7-9bfdb70053bf" containerID="a3411fdb6fb045dd740bc206d03316890fe8436b1cbe028bb72e2c89b8ccd640" exitCode=0 Jul 03 10:31:30.379193 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.378845 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" event={"ID":"e3069f5d-446c-459e-b5a7-9bfdb70053bf","Type":"ContainerDied","Data":"a3411fdb6fb045dd740bc206d03316890fe8436b1cbe028bb72e2c89b8ccd640"} Jul 03 10:31:30.468498 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.468472 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:30.522163 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522052 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-data\") pod \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " Jul 03 10:31:30.522163 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522153 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istiod-ca-cert\") pod \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " Jul 03 10:31:30.522403 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522181 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-socket\") pod \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " Jul 03 10:31:30.522403 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522201 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp5mr\" (UniqueName: \"kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-kube-api-access-mp5mr\") pod \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " Jul 03 10:31:30.522403 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522239 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-certs\") pod \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " Jul 03 10:31:30.522403 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522272 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-token\") pod \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " Jul 03 10:31:30.522403 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522300 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-credential-socket\") pod \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " Jul 03 10:31:30.522403 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522326 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-envoy\") pod \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " Jul 03 10:31:30.522403 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522357 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-podinfo\") pod \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\" (UID: \"e3069f5d-446c-459e-b5a7-9bfdb70053bf\") " Jul 03 10:31:30.522765 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522480 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e3069f5d-446c-459e-b5a7-9bfdb70053bf" (UID: "e3069f5d-446c-459e-b5a7-9bfdb70053bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:31:30.522765 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522579 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e3069f5d-446c-459e-b5a7-9bfdb70053bf" (UID: "e3069f5d-446c-459e-b5a7-9bfdb70053bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:31:30.522765 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522673 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:30.522765 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522692 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:30.522765 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522752 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e3069f5d-446c-459e-b5a7-9bfdb70053bf" (UID: "e3069f5d-446c-459e-b5a7-9bfdb70053bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:31:30.523017 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522844 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e3069f5d-446c-459e-b5a7-9bfdb70053bf" (UID: "e3069f5d-446c-459e-b5a7-9bfdb70053bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:31:30.523017 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.522943 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "e3069f5d-446c-459e-b5a7-9bfdb70053bf" (UID: "e3069f5d-446c-459e-b5a7-9bfdb70053bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:31:30.524857 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.524835 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e3069f5d-446c-459e-b5a7-9bfdb70053bf" (UID: "e3069f5d-446c-459e-b5a7-9bfdb70053bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:31:30.524977 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.524908 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-kube-api-access-mp5mr" (OuterVolumeSpecName: "kube-api-access-mp5mr") pod "e3069f5d-446c-459e-b5a7-9bfdb70053bf" (UID: "e3069f5d-446c-459e-b5a7-9bfdb70053bf"). InnerVolumeSpecName "kube-api-access-mp5mr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:31:30.524977 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.524965 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "e3069f5d-446c-459e-b5a7-9bfdb70053bf" (UID: "e3069f5d-446c-459e-b5a7-9bfdb70053bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:31:30.525081 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.525057 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e3069f5d-446c-459e-b5a7-9bfdb70053bf" (UID: "e3069f5d-446c-459e-b5a7-9bfdb70053bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:31:30.623760 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.623727 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp5mr\" (UniqueName: \"kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-kube-api-access-mp5mr\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:30.623760 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.623758 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:30.623760 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.623774 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:30.624028 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.623786 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:30.624028 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.623799 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:30.624028 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.623809 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:30.624028 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:30.623820 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3069f5d-446c-459e-b5a7-9bfdb70053bf-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:31.385254 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.385222 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" Jul 03 10:31:31.385254 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.385250 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4" event={"ID":"e3069f5d-446c-459e-b5a7-9bfdb70053bf","Type":"ContainerDied","Data":"e039c0a282e53dc865c73ee2d07e7d6b0da68f3f4a0cc2dd9001f4df31e8f705"} Jul 03 10:31:31.385798 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.385316 2577 scope.go:117] "RemoveContainer" containerID="a3411fdb6fb045dd740bc206d03316890fe8436b1cbe028bb72e2c89b8ccd640" Jul 03 10:31:31.412586 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.412553 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4"] Jul 03 10:31:31.415115 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.415071 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-322e29fb-openshift-default-d498bcdc7-69kp4"] Jul 03 10:31:31.582964 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.582929 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb"] Jul 03 10:31:31.583376 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.583358 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7bfdf78-3341-43b8-88e2-04e64d2412e7" containerName="maas-api" Jul 03 10:31:31.583465 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.583378 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bfdf78-3341-43b8-88e2-04e64d2412e7" containerName="maas-api" Jul 03 10:31:31.583465 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.583396 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3069f5d-446c-459e-b5a7-9bfdb70053bf" containerName="istio-proxy" Jul 03 10:31:31.583465 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.583404 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3069f5d-446c-459e-b5a7-9bfdb70053bf" containerName="istio-proxy" Jul 03 10:31:31.583626 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.583501 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7bfdf78-3341-43b8-88e2-04e64d2412e7" containerName="maas-api" Jul 03 10:31:31.583626 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.583516 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3069f5d-446c-459e-b5a7-9bfdb70053bf" containerName="istio-proxy" Jul 03 10:31:31.587945 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.587922 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.594783 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.594762 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fe5f6fa6-openshift-default-dockercfg-9d5pl\"" Jul 03 10:31:31.608003 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.607976 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb"] Jul 03 10:31:31.632464 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.632435 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4a7e868-0372-49fb-8605-28adca7271af-istiod-ca-cert\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.632629 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.632483 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-credential-socket\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.632629 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.632555 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-istio-token\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.632629 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.632604 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-certs\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.632799 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.632643 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-data\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.632799 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.632669 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbrh2\" (UniqueName: \"kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-kube-api-access-mbrh2\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.632799 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.632692 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4a7e868-0372-49fb-8605-28adca7271af-istio-podinfo\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.632799 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.632727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-socket\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.632799 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.632776 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-envoy\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.734158 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734087 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbrh2\" (UniqueName: \"kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-kube-api-access-mbrh2\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.734158 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4a7e868-0372-49fb-8605-28adca7271af-istio-podinfo\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.734430 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734193 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-socket\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.734430 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-envoy\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.734430 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734290 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4a7e868-0372-49fb-8605-28adca7271af-istiod-ca-cert\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.734430 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734332 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-credential-socket\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.734430 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-istio-token\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.735079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-certs\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.735079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-data\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.735079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734604 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-socket\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.735079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734903 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-data\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.735079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734935 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-credential-socket\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.735079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.734990 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-certs\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.735327 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.735144 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4a7e868-0372-49fb-8605-28adca7271af-istiod-ca-cert\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.737153 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.737127 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-envoy\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.737276 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.737258 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4a7e868-0372-49fb-8605-28adca7271af-istio-podinfo\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.743538 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.743518 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-istio-token\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.744053 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.744032 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbrh2\" (UniqueName: \"kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-kube-api-access-mbrh2\") pod \"e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:31.898045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:31.898014 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:32.051396 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.051365 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb"] Jul 03 10:31:32.052719 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:31:32.052696 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a7e868_0372_49fb_8605_28adca7271af.slice/crio-420097fab53d256894d52ff99ba5548885716d63772353b28b49703c0f4523eb WatchSource:0}: Error finding container 420097fab53d256894d52ff99ba5548885716d63772353b28b49703c0f4523eb: Status 404 returned error can't find the container with id 420097fab53d256894d52ff99ba5548885716d63772353b28b49703c0f4523eb Jul 03 10:31:32.054907 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.054875 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:31:32.055056 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.054957 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:31:32.055056 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.055003 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:31:32.392642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.392528 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" event={"ID":"a4a7e868-0372-49fb-8605-28adca7271af","Type":"ContainerStarted","Data":"20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648"} Jul 03 10:31:32.392642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.392594 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" event={"ID":"a4a7e868-0372-49fb-8605-28adca7271af","Type":"ContainerStarted","Data":"420097fab53d256894d52ff99ba5548885716d63772353b28b49703c0f4523eb"} Jul 03 10:31:32.429787 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.429734 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" podStartSLOduration=1.429716197 podStartE2EDuration="1.429716197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:31:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:31:32.428282039 +0000 UTC m=+1570.494357241" watchObservedRunningTime="2026-07-03 10:31:32.429716197 +0000 UTC m=+1570.495791444" Jul 03 10:31:32.476957 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.476920 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3069f5d-446c-459e-b5a7-9bfdb70053bf" path="/var/lib/kubelet/pods/e3069f5d-446c-459e-b5a7-9bfdb70053bf/volumes" Jul 03 10:31:32.898277 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.898228 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:32.988466 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.988433 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn"] Jul 03 10:31:32.992410 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.992385 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:32.998016 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:32.997992 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-fe5f6fa6-serving-cert\"" Jul 03 10:31:33.021865 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.021828 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn"] Jul 03 10:31:33.045875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.045842 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-openshift-service-ca\") pod \"maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:33.046045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.045887 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-maas-api-tls\") pod \"maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:33.046045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.045906 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v77sf\" (UniqueName: \"kubernetes.io/projected/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-kube-api-access-v77sf\") pod \"maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:33.147131 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.147075 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-openshift-service-ca\") pod \"maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:33.147352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.147143 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-maas-api-tls\") pod \"maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:33.147352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.147166 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v77sf\" (UniqueName: \"kubernetes.io/projected/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-kube-api-access-v77sf\") pod \"maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:33.147797 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.147768 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-openshift-service-ca\") pod \"maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:33.149607 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.149559 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-maas-api-tls\") pod \"maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:33.156209 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.156182 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v77sf\" (UniqueName: \"kubernetes.io/projected/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-kube-api-access-v77sf\") pod \"maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:33.303623 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.303586 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:33.447897 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.447867 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn"] Jul 03 10:31:33.449289 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:31:33.449258 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab1f4a8b_bccc_42d3_ae91_fed90d8ab80a.slice/crio-a319d3255dcb073ea9e70d2fb0209a31ea460b1424c5202ba556a3fb5d195413 WatchSource:0}: Error finding container a319d3255dcb073ea9e70d2fb0209a31ea460b1424c5202ba556a3fb5d195413: Status 404 returned error can't find the container with id a319d3255dcb073ea9e70d2fb0209a31ea460b1424c5202ba556a3fb5d195413 Jul 03 10:31:33.904425 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:33.904396 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:34.407254 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:34.407220 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" event={"ID":"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a","Type":"ContainerStarted","Data":"5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077"} Jul 03 10:31:34.407434 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:34.407274 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" event={"ID":"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a","Type":"ContainerStarted","Data":"a319d3255dcb073ea9e70d2fb0209a31ea460b1424c5202ba556a3fb5d195413"} Jul 03 10:31:34.407524 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:34.407433 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:34.407524 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:34.407484 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:34.408526 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:34.408507 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:34.443316 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:34.443257 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" podStartSLOduration=2.052243366 podStartE2EDuration="2.443239694s" totalImagesPullingTime="390.996328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:31:34.440537158 +0000 UTC m=+1572.506612350" watchObservedRunningTime="2026-07-03 10:31:34.443239694 +0000 UTC m=+1572.509314885" Jul 03 10:31:38.368684 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.366828 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn"] Jul 03 10:31:38.368684 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.367088 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" podUID="ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a" containerName="maas-api" containerID="cri-o://5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077" gracePeriod=30 Jul 03 10:31:38.374805 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.374778 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:38.619533 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.619464 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:38.697172 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.697135 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v77sf\" (UniqueName: \"kubernetes.io/projected/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-kube-api-access-v77sf\") pod \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " Jul 03 10:31:38.697374 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.697189 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-openshift-service-ca\") pod \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " Jul 03 10:31:38.697374 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.697305 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-maas-api-tls\") pod \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\" (UID: \"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a\") " Jul 03 10:31:38.697564 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.697539 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a" (UID: "ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:31:38.699338 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.699315 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-kube-api-access-v77sf" (OuterVolumeSpecName: "kube-api-access-v77sf") pod "ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a" (UID: "ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a"). InnerVolumeSpecName "kube-api-access-v77sf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:31:38.699430 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.699382 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a" (UID: "ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:31:38.798660 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.798620 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:38.798660 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.798653 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:38.798660 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:38.798662 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v77sf\" (UniqueName: \"kubernetes.io/projected/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a-kube-api-access-v77sf\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:39.435528 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:39.435489 2577 generic.go:356] "Generic (PLEG): container finished" podID="ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a" containerID="5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077" exitCode=0 Jul 03 10:31:39.436007 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:39.435536 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" Jul 03 10:31:39.436007 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:39.435576 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" event={"ID":"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a","Type":"ContainerDied","Data":"5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077"} Jul 03 10:31:39.436007 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:39.435640 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn" event={"ID":"ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a","Type":"ContainerDied","Data":"a319d3255dcb073ea9e70d2fb0209a31ea460b1424c5202ba556a3fb5d195413"} Jul 03 10:31:39.436007 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:39.435675 2577 scope.go:117] "RemoveContainer" containerID="5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077" Jul 03 10:31:39.445952 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:39.445934 2577 scope.go:117] "RemoveContainer" containerID="5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077" Jul 03 10:31:39.446278 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:31:39.446259 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077\": container with ID starting with 5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077 not found: ID does not exist" containerID="5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077" Jul 03 10:31:39.446360 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:39.446285 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077"} err="failed to get container status \"5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077\": rpc error: code = NotFound desc = could not find container \"5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077\": container with ID starting with 5f6993828e6b38a39756be6c6386b0175079761dbee7110624aeabce10079077 not found: ID does not exist" Jul 03 10:31:39.462824 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:39.462791 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn"] Jul 03 10:31:39.467567 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:39.467541 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-fe5f6fa6-6dc9df65-qjksn"] Jul 03 10:31:40.471990 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:40.471957 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a" path="/var/lib/kubelet/pods/ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a/volumes" Jul 03 10:31:44.222971 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:44.222939 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb"] Jul 03 10:31:44.223538 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:44.223163 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" podUID="a4a7e868-0372-49fb-8605-28adca7271af" containerName="istio-proxy" containerID="cri-o://20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648" gracePeriod=30 Jul 03 10:31:49.483144 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.483108 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:49.493635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.493555 2577 generic.go:356] "Generic (PLEG): container finished" podID="a4a7e868-0372-49fb-8605-28adca7271af" containerID="20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648" exitCode=0 Jul 03 10:31:49.493635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.493600 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" Jul 03 10:31:49.493805 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.493630 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" event={"ID":"a4a7e868-0372-49fb-8605-28adca7271af","Type":"ContainerDied","Data":"20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648"} Jul 03 10:31:49.493805 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.493685 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" event={"ID":"a4a7e868-0372-49fb-8605-28adca7271af","Type":"ContainerDied","Data":"420097fab53d256894d52ff99ba5548885716d63772353b28b49703c0f4523eb"} Jul 03 10:31:49.493805 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.493711 2577 scope.go:117] "RemoveContainer" containerID="20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648" Jul 03 10:31:49.502598 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.502576 2577 scope.go:117] "RemoveContainer" containerID="20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648" Jul 03 10:31:49.502889 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:31:49.502870 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648\": container with ID starting with 20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648 not found: ID does not exist" containerID="20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648" Jul 03 10:31:49.502931 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.502899 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648"} err="failed to get container status \"20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648\": rpc error: code = NotFound desc = could not find container \"20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648\": container with ID starting with 20bd96ed94107b4738125cbb62b5f01b8b91b54650069a44e094b164fb934648 not found: ID does not exist" Jul 03 10:31:49.594885 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.594845 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-socket\") pod \"a4a7e868-0372-49fb-8605-28adca7271af\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " Jul 03 10:31:49.595076 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.594900 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-certs\") pod \"a4a7e868-0372-49fb-8605-28adca7271af\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " Jul 03 10:31:49.595076 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.594932 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-data\") pod \"a4a7e868-0372-49fb-8605-28adca7271af\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " Jul 03 10:31:49.595076 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.594972 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-credential-socket\") pod \"a4a7e868-0372-49fb-8605-28adca7271af\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " Jul 03 10:31:49.595076 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595015 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4a7e868-0372-49fb-8605-28adca7271af-istiod-ca-cert\") pod \"a4a7e868-0372-49fb-8605-28adca7271af\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " Jul 03 10:31:49.595076 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595075 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-envoy\") pod \"a4a7e868-0372-49fb-8605-28adca7271af\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " Jul 03 10:31:49.595364 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595199 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4a7e868-0372-49fb-8605-28adca7271af-istio-podinfo\") pod \"a4a7e868-0372-49fb-8605-28adca7271af\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " Jul 03 10:31:49.595364 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595246 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-istio-token\") pod \"a4a7e868-0372-49fb-8605-28adca7271af\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " Jul 03 10:31:49.595364 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595316 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbrh2\" (UniqueName: \"kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-kube-api-access-mbrh2\") pod \"a4a7e868-0372-49fb-8605-28adca7271af\" (UID: \"a4a7e868-0372-49fb-8605-28adca7271af\") " Jul 03 10:31:49.595364 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595238 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a4a7e868-0372-49fb-8605-28adca7271af" (UID: "a4a7e868-0372-49fb-8605-28adca7271af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:31:49.595364 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595249 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a4a7e868-0372-49fb-8605-28adca7271af" (UID: "a4a7e868-0372-49fb-8605-28adca7271af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:31:49.595611 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595442 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a4a7e868-0372-49fb-8605-28adca7271af" (UID: "a4a7e868-0372-49fb-8605-28adca7271af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:31:49.595611 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595552 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a7e868-0372-49fb-8605-28adca7271af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a4a7e868-0372-49fb-8605-28adca7271af" (UID: "a4a7e868-0372-49fb-8605-28adca7271af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:31:49.595772 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595614 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-data" (OuterVolumeSpecName: "istio-data") pod "a4a7e868-0372-49fb-8605-28adca7271af" (UID: "a4a7e868-0372-49fb-8605-28adca7271af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:31:49.595772 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595652 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:49.595772 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595670 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:49.595772 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595684 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:49.595772 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.595699 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4a7e868-0372-49fb-8605-28adca7271af-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:49.597997 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.597971 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4a7e868-0372-49fb-8605-28adca7271af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a4a7e868-0372-49fb-8605-28adca7271af" (UID: "a4a7e868-0372-49fb-8605-28adca7271af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:31:49.597997 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.597992 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-kube-api-access-mbrh2" (OuterVolumeSpecName: "kube-api-access-mbrh2") pod "a4a7e868-0372-49fb-8605-28adca7271af" (UID: "a4a7e868-0372-49fb-8605-28adca7271af"). InnerVolumeSpecName "kube-api-access-mbrh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:31:49.597997 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.597975 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4a7e868-0372-49fb-8605-28adca7271af" (UID: "a4a7e868-0372-49fb-8605-28adca7271af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:31:49.597997 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.597978 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a4a7e868-0372-49fb-8605-28adca7271af" (UID: "a4a7e868-0372-49fb-8605-28adca7271af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:31:49.696763 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.696722 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4a7e868-0372-49fb-8605-28adca7271af-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:49.696763 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.696756 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:49.696763 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.696766 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbrh2\" (UniqueName: \"kubernetes.io/projected/a4a7e868-0372-49fb-8605-28adca7271af-kube-api-access-mbrh2\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:49.697006 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.696777 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:49.697006 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.696786 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4a7e868-0372-49fb-8605-28adca7271af-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:49.819342 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.819302 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb"] Jul 03 10:31:49.830439 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:49.830408 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb"] Jul 03 10:31:50.409162 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.409119 2577 patch_prober.go:28] interesting pod/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 03 10:31:50.409352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.409192 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-fe5f6fa6-openshift-default-86b67859b5-xdsjb" podUID="a4a7e868-0372-49fb-8605-28adca7271af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 03 10:31:50.475839 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.475805 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4a7e868-0372-49fb-8605-28adca7271af" path="/var/lib/kubelet/pods/a4a7e868-0372-49fb-8605-28adca7271af/volumes" Jul 03 10:31:50.750042 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.749955 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq"] Jul 03 10:31:50.750587 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.750376 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a7e868-0372-49fb-8605-28adca7271af" containerName="istio-proxy" Jul 03 10:31:50.750587 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.750391 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a7e868-0372-49fb-8605-28adca7271af" containerName="istio-proxy" Jul 03 10:31:50.750587 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.750400 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a" containerName="maas-api" Jul 03 10:31:50.750587 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.750407 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a" containerName="maas-api" Jul 03 10:31:50.750587 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.750487 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a7e868-0372-49fb-8605-28adca7271af" containerName="istio-proxy" Jul 03 10:31:50.750587 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.750499 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab1f4a8b-bccc-42d3-ae91-fed90d8ab80a" containerName="maas-api" Jul 03 10:31:50.754928 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.754898 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.758845 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.758820 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-f29489ac-openshift-default-dockercfg-gkrkx\"" Jul 03 10:31:50.779254 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.779224 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq"] Jul 03 10:31:50.807671 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.807634 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-credential-socket\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.807856 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.807678 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4jjf\" (UniqueName: \"kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-kube-api-access-q4jjf\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.807856 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.807699 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-token\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.807856 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.807758 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-envoy\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.808040 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.807866 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-socket\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.808040 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.807932 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-data\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.808040 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.807963 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-certs\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.808040 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.807991 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-podinfo\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.808040 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.808032 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed6e4dfb-2782-40d5-b032-5811476231b1-istiod-ca-cert\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.908819 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.908782 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4jjf\" (UniqueName: \"kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-kube-api-access-q4jjf\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.908819 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.908821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-token\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.908845 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-envoy\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.908869 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-socket\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.908903 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-data\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.908931 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-certs\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.908960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-podinfo\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.908987 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed6e4dfb-2782-40d5-b032-5811476231b1-istiod-ca-cert\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909082 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.909064 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-credential-socket\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909475 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.909362 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-data\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909777 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.909750 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed6e4dfb-2782-40d5-b032-5811476231b1-istiod-ca-cert\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909777 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.909771 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-socket\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909953 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.909829 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-certs\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.909953 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.909848 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-credential-socket\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.911381 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.911363 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-envoy\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.911542 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.911524 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-podinfo\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.919726 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.919702 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4jjf\" (UniqueName: \"kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-kube-api-access-q4jjf\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:50.920197 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:50.920177 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-token\") pod \"e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:51.067229 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:51.067195 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:51.220465 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:51.220439 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq"] Jul 03 10:31:51.222111 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:31:51.222069 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded6e4dfb_2782_40d5_b032_5811476231b1.slice/crio-fef0b4ac3adbf4abf38498435f1c4ba3288e861e0191046844d3fca4c523f1ae WatchSource:0}: Error finding container fef0b4ac3adbf4abf38498435f1c4ba3288e861e0191046844d3fca4c523f1ae: Status 404 returned error can't find the container with id fef0b4ac3adbf4abf38498435f1c4ba3288e861e0191046844d3fca4c523f1ae Jul 03 10:31:51.224309 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:51.224275 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:31:51.224396 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:51.224343 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:31:51.224396 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:51.224376 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:31:51.507562 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:51.507515 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" event={"ID":"ed6e4dfb-2782-40d5-b032-5811476231b1","Type":"ContainerStarted","Data":"9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d"} Jul 03 10:31:51.507770 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:51.507578 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" event={"ID":"ed6e4dfb-2782-40d5-b032-5811476231b1","Type":"ContainerStarted","Data":"fef0b4ac3adbf4abf38498435f1c4ba3288e861e0191046844d3fca4c523f1ae"} Jul 03 10:31:51.560237 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:51.560176 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" podStartSLOduration=1.560159554 podStartE2EDuration="1.560159554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:31:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:31:51.55673229 +0000 UTC m=+1589.622807482" watchObservedRunningTime="2026-07-03 10:31:51.560159554 +0000 UTC m=+1589.626234746" Jul 03 10:31:52.068335 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.068300 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:52.073482 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.073404 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:52.086510 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.086477 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr"] Jul 03 10:31:52.090131 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.090107 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:52.094467 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.094435 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-f29489ac-serving-cert\"" Jul 03 10:31:52.116155 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.116119 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr"] Jul 03 10:31:52.222783 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.222742 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcdxv\" (UniqueName: \"kubernetes.io/projected/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-kube-api-access-lcdxv\") pod \"maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:52.222977 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.222797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-openshift-service-ca\") pod \"maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:52.222977 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.222874 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-maas-api-tls\") pod \"maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:52.323799 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.323686 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdxv\" (UniqueName: \"kubernetes.io/projected/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-kube-api-access-lcdxv\") pod \"maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:52.323799 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.323755 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-openshift-service-ca\") pod \"maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:52.323799 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.323803 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-maas-api-tls\") pod \"maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:52.324039 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:31:52.323963 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-f29489ac-serving-cert: secret "maas-api-e2e-derive-f29489ac-serving-cert" not found Jul 03 10:31:52.324039 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:31:52.324036 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-maas-api-tls podName:d713a0f7-1b6d-4214-bf50-cc4e4025d4db nodeName:}" failed. No retries permitted until 2026-07-03 10:31:52.824016078 +0000 UTC m=+1590.890091247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-maas-api-tls") pod "maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" (UID: "d713a0f7-1b6d-4214-bf50-cc4e4025d4db") : secret "maas-api-e2e-derive-f29489ac-serving-cert" not found Jul 03 10:31:52.324516 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.324496 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-openshift-service-ca\") pod \"maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:52.337145 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.337089 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcdxv\" (UniqueName: \"kubernetes.io/projected/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-kube-api-access-lcdxv\") pod \"maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:52.513450 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.513413 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:52.514570 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.514552 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:31:52.828290 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.828251 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-maas-api-tls\") pod \"maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:52.830781 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:52.830753 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-maas-api-tls\") pod \"maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:53.001197 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:53.001156 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:53.150192 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:53.150165 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr"] Jul 03 10:31:53.152608 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:31:53.152570 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd713a0f7_1b6d_4214_bf50_cc4e4025d4db.slice/crio-7643286cc4607eadc3cca4d5838f8ea0e2a5327c940a0a1fb6fca73bf1de2a51 WatchSource:0}: Error finding container 7643286cc4607eadc3cca4d5838f8ea0e2a5327c940a0a1fb6fca73bf1de2a51: Status 404 returned error can't find the container with id 7643286cc4607eadc3cca4d5838f8ea0e2a5327c940a0a1fb6fca73bf1de2a51 Jul 03 10:31:53.519507 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:53.519473 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" event={"ID":"d713a0f7-1b6d-4214-bf50-cc4e4025d4db","Type":"ContainerStarted","Data":"7643286cc4607eadc3cca4d5838f8ea0e2a5327c940a0a1fb6fca73bf1de2a51"} Jul 03 10:31:54.527403 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:54.527361 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" event={"ID":"d713a0f7-1b6d-4214-bf50-cc4e4025d4db","Type":"ContainerStarted","Data":"85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c"} Jul 03 10:31:54.571045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:54.570985 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" podStartSLOduration=2.172117437 podStartE2EDuration="2.570968086s" totalImagesPullingTime="398.850649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:31:54.570463095 +0000 UTC m=+1592.636538287" watchObservedRunningTime="2026-07-03 10:31:54.570968086 +0000 UTC m=+1592.637043278" Jul 03 10:31:55.534420 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:55.534377 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:56.564139 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:56.564087 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq"] Jul 03 10:31:56.564547 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:56.564308 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" podUID="ed6e4dfb-2782-40d5-b032-5811476231b1" containerName="istio-proxy" containerID="cri-o://9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d" gracePeriod=30 Jul 03 10:31:56.846177 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:56.844550 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr"] Jul 03 10:31:57.550119 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.549558 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" podUID="d713a0f7-1b6d-4214-bf50-cc4e4025d4db" containerName="maas-api" containerID="cri-o://85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c" gracePeriod=30 Jul 03 10:31:57.816000 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.815975 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:57.880113 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.880075 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-openshift-service-ca\") pod \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " Jul 03 10:31:57.880303 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.880204 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcdxv\" (UniqueName: \"kubernetes.io/projected/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-kube-api-access-lcdxv\") pod \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " Jul 03 10:31:57.880303 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.880244 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-maas-api-tls\") pod \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\" (UID: \"d713a0f7-1b6d-4214-bf50-cc4e4025d4db\") " Jul 03 10:31:57.880421 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.880402 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d713a0f7-1b6d-4214-bf50-cc4e4025d4db" (UID: "d713a0f7-1b6d-4214-bf50-cc4e4025d4db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:31:57.882325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.882296 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d713a0f7-1b6d-4214-bf50-cc4e4025d4db" (UID: "d713a0f7-1b6d-4214-bf50-cc4e4025d4db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:31:57.882502 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.882483 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-kube-api-access-lcdxv" (OuterVolumeSpecName: "kube-api-access-lcdxv") pod "d713a0f7-1b6d-4214-bf50-cc4e4025d4db" (UID: "d713a0f7-1b6d-4214-bf50-cc4e4025d4db"). InnerVolumeSpecName "kube-api-access-lcdxv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:31:57.981714 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.981679 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:57.981714 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.981709 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:57.981714 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:57.981718 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcdxv\" (UniqueName: \"kubernetes.io/projected/d713a0f7-1b6d-4214-bf50-cc4e4025d4db-kube-api-access-lcdxv\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:31:58.558715 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:58.558677 2577 generic.go:356] "Generic (PLEG): container finished" podID="d713a0f7-1b6d-4214-bf50-cc4e4025d4db" containerID="85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c" exitCode=0 Jul 03 10:31:58.558882 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:58.558736 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" Jul 03 10:31:58.558882 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:58.558760 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" event={"ID":"d713a0f7-1b6d-4214-bf50-cc4e4025d4db","Type":"ContainerDied","Data":"85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c"} Jul 03 10:31:58.558882 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:58.558843 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr" event={"ID":"d713a0f7-1b6d-4214-bf50-cc4e4025d4db","Type":"ContainerDied","Data":"7643286cc4607eadc3cca4d5838f8ea0e2a5327c940a0a1fb6fca73bf1de2a51"} Jul 03 10:31:58.558882 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:58.558869 2577 scope.go:117] "RemoveContainer" containerID="85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c" Jul 03 10:31:58.567965 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:58.567953 2577 scope.go:117] "RemoveContainer" containerID="85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c" Jul 03 10:31:58.568294 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:31:58.568268 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c\": container with ID starting with 85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c not found: ID does not exist" containerID="85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c" Jul 03 10:31:58.568383 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:58.568306 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c"} err="failed to get container status \"85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c\": rpc error: code = NotFound desc = could not find container \"85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c\": container with ID starting with 85208fca60fa2aa880e814e246c50fd7ef029656bbbff27f8d67210e8caa5c4c not found: ID does not exist" Jul 03 10:31:58.577334 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:58.577303 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr"] Jul 03 10:31:58.581800 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:31:58.581775 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-f29489ac-747d47d7f5-x2klr"] Jul 03 10:32:00.479357 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:00.479321 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d713a0f7-1b6d-4214-bf50-cc4e4025d4db" path="/var/lib/kubelet/pods/d713a0f7-1b6d-4214-bf50-cc4e4025d4db/volumes" Jul 03 10:32:02.118712 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.118686 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:32:02.219450 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219356 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-podinfo\") pod \"ed6e4dfb-2782-40d5-b032-5811476231b1\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " Jul 03 10:32:02.219450 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219405 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-credential-socket\") pod \"ed6e4dfb-2782-40d5-b032-5811476231b1\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " Jul 03 10:32:02.219450 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219428 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-envoy\") pod \"ed6e4dfb-2782-40d5-b032-5811476231b1\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " Jul 03 10:32:02.219450 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219449 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-certs\") pod \"ed6e4dfb-2782-40d5-b032-5811476231b1\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " Jul 03 10:32:02.219870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219480 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-token\") pod \"ed6e4dfb-2782-40d5-b032-5811476231b1\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " Jul 03 10:32:02.219870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219514 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-socket\") pod \"ed6e4dfb-2782-40d5-b032-5811476231b1\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " Jul 03 10:32:02.219870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219551 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed6e4dfb-2782-40d5-b032-5811476231b1-istiod-ca-cert\") pod \"ed6e4dfb-2782-40d5-b032-5811476231b1\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " Jul 03 10:32:02.219870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219622 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-data\") pod \"ed6e4dfb-2782-40d5-b032-5811476231b1\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " Jul 03 10:32:02.219870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219647 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4jjf\" (UniqueName: \"kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-kube-api-access-q4jjf\") pod \"ed6e4dfb-2782-40d5-b032-5811476231b1\" (UID: \"ed6e4dfb-2782-40d5-b032-5811476231b1\") " Jul 03 10:32:02.219870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219850 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed6e4dfb-2782-40d5-b032-5811476231b1" (UID: "ed6e4dfb-2782-40d5-b032-5811476231b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:32:02.220239 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219871 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed6e4dfb-2782-40d5-b032-5811476231b1" (UID: "ed6e4dfb-2782-40d5-b032-5811476231b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:32:02.220239 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.219981 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:32:02.220239 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.220003 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:32:02.220239 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.220132 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed6e4dfb-2782-40d5-b032-5811476231b1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed6e4dfb-2782-40d5-b032-5811476231b1" (UID: "ed6e4dfb-2782-40d5-b032-5811476231b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:32:02.220239 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.220158 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed6e4dfb-2782-40d5-b032-5811476231b1" (UID: "ed6e4dfb-2782-40d5-b032-5811476231b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:32:02.220817 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.220793 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed6e4dfb-2782-40d5-b032-5811476231b1" (UID: "ed6e4dfb-2782-40d5-b032-5811476231b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:32:02.221891 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.221854 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed6e4dfb-2782-40d5-b032-5811476231b1" (UID: "ed6e4dfb-2782-40d5-b032-5811476231b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:32:02.222022 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.221899 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-kube-api-access-q4jjf" (OuterVolumeSpecName: "kube-api-access-q4jjf") pod "ed6e4dfb-2782-40d5-b032-5811476231b1" (UID: "ed6e4dfb-2782-40d5-b032-5811476231b1"). InnerVolumeSpecName "kube-api-access-q4jjf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:32:02.222089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.222068 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed6e4dfb-2782-40d5-b032-5811476231b1" (UID: "ed6e4dfb-2782-40d5-b032-5811476231b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:32:02.222089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.222075 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed6e4dfb-2782-40d5-b032-5811476231b1" (UID: "ed6e4dfb-2782-40d5-b032-5811476231b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:32:02.321041 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.321000 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:32:02.321041 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.321033 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:32:02.321041 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.321041 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:32:02.321041 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.321050 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed6e4dfb-2782-40d5-b032-5811476231b1-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:32:02.321341 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.321060 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:32:02.321341 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.321068 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4jjf\" (UniqueName: \"kubernetes.io/projected/ed6e4dfb-2782-40d5-b032-5811476231b1-kube-api-access-q4jjf\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:32:02.321341 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.321076 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed6e4dfb-2782-40d5-b032-5811476231b1-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:32:02.584294 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.584259 2577 generic.go:356] "Generic (PLEG): container finished" podID="ed6e4dfb-2782-40d5-b032-5811476231b1" containerID="9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d" exitCode=0 Jul 03 10:32:02.584472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.584306 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" Jul 03 10:32:02.584472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.584339 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" event={"ID":"ed6e4dfb-2782-40d5-b032-5811476231b1","Type":"ContainerDied","Data":"9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d"} Jul 03 10:32:02.584472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.584392 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq" event={"ID":"ed6e4dfb-2782-40d5-b032-5811476231b1","Type":"ContainerDied","Data":"fef0b4ac3adbf4abf38498435f1c4ba3288e861e0191046844d3fca4c523f1ae"} Jul 03 10:32:02.584472 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.584412 2577 scope.go:117] "RemoveContainer" containerID="9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d" Jul 03 10:32:02.593506 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.593484 2577 scope.go:117] "RemoveContainer" containerID="9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d" Jul 03 10:32:02.593835 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:32:02.593816 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d\": container with ID starting with 9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d not found: ID does not exist" containerID="9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d" Jul 03 10:32:02.593899 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.593851 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d"} err="failed to get container status \"9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d\": rpc error: code = NotFound desc = could not find container \"9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d\": container with ID starting with 9c68526b6cde72c4ef07a9af890ddf6659cbbd895316eca926ab26b193a3332d not found: ID does not exist" Jul 03 10:32:02.607336 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.607303 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq"] Jul 03 10:32:02.611828 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:02.611798 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-f29489ac-openshift-default-5984b84f75-ps4kq"] Jul 03 10:32:04.472064 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:32:04.472030 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed6e4dfb-2782-40d5-b032-5811476231b1" path="/var/lib/kubelet/pods/ed6e4dfb-2782-40d5-b032-5811476231b1/volumes" Jul 03 10:33:40.213575 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.213535 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr"] Jul 03 10:33:40.214005 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.213883 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d713a0f7-1b6d-4214-bf50-cc4e4025d4db" containerName="maas-api" Jul 03 10:33:40.214005 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.213894 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d713a0f7-1b6d-4214-bf50-cc4e4025d4db" containerName="maas-api" Jul 03 10:33:40.214005 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.213915 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed6e4dfb-2782-40d5-b032-5811476231b1" containerName="istio-proxy" Jul 03 10:33:40.214005 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.213921 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed6e4dfb-2782-40d5-b032-5811476231b1" containerName="istio-proxy" Jul 03 10:33:40.214005 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.213978 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed6e4dfb-2782-40d5-b032-5811476231b1" containerName="istio-proxy" Jul 03 10:33:40.214005 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.213986 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d713a0f7-1b6d-4214-bf50-cc4e4025d4db" containerName="maas-api" Jul 03 10:33:40.216950 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.216933 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.219937 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.219914 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0d436a63-openshift-default-dockercfg-7fvn9\"" Jul 03 10:33:40.251247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.251213 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr"] Jul 03 10:33:40.263804 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.263774 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jndh2\" (UniqueName: \"kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-kube-api-access-jndh2\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.264011 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.263822 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-envoy\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.264011 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.263911 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-token\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.264011 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.263941 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-socket\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.264011 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.263958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-credential-socket\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.264011 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.263972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-podinfo\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.264011 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.264005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-data\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.264267 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.264045 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istiod-ca-cert\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.264267 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.264075 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-certs\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.365469 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.365424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-envoy\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.365646 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.365513 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-token\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.365646 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.365549 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-socket\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.365646 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.365574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-credential-socket\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.365646 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.365597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-podinfo\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.365646 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.365635 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-data\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.365963 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.365675 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istiod-ca-cert\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.365963 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.365723 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-certs\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.365963 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.365774 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jndh2\" (UniqueName: \"kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-kube-api-access-jndh2\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.366189 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.366053 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-credential-socket\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.366189 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.366077 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-data\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.366420 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.366399 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-certs\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.366560 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.366397 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-socket\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.366697 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.366673 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istiod-ca-cert\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.367986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.367968 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-envoy\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.368071 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.368046 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-podinfo\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.374992 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.374973 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-token\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.375087 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.375046 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jndh2\" (UniqueName: \"kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-kube-api-access-jndh2\") pod \"e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.527076 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.526987 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:40.668894 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.668865 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr"] Jul 03 10:33:40.670183 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:33:40.670150 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb098c77_87b0_462d_a5b7_f5e7dcd5ce43.slice/crio-a616d6b8f67808990d28c5eb318a2a1dcdec90aa7b4c9674b75105001b52e73d WatchSource:0}: Error finding container a616d6b8f67808990d28c5eb318a2a1dcdec90aa7b4c9674b75105001b52e73d: Status 404 returned error can't find the container with id a616d6b8f67808990d28c5eb318a2a1dcdec90aa7b4c9674b75105001b52e73d Jul 03 10:33:40.672258 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.672225 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:33:40.672336 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.672296 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:33:40.672336 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:40.672323 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:33:41.098013 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:41.097972 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" event={"ID":"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43","Type":"ContainerStarted","Data":"e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed"} Jul 03 10:33:41.098227 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:41.098053 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" event={"ID":"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43","Type":"ContainerStarted","Data":"a616d6b8f67808990d28c5eb318a2a1dcdec90aa7b4c9674b75105001b52e73d"} Jul 03 10:33:41.128399 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:41.128341 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" podStartSLOduration=1.128325678 podStartE2EDuration="1.128325678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:33:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:33:41.125792703 +0000 UTC m=+1699.191867898" watchObservedRunningTime="2026-07-03 10:33:41.128325678 +0000 UTC m=+1699.194400870" Jul 03 10:33:41.527955 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:41.527920 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:41.533329 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:41.533305 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:42.103495 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:42.103462 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:42.104370 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:42.104354 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:43.389737 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.389701 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb"] Jul 03 10:33:43.393664 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.393645 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.396720 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.396696 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-0d436a63-serving-cert\"" Jul 03 10:33:43.405845 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.405820 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb"] Jul 03 10:33:43.496089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.496055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghhmg\" (UniqueName: \"kubernetes.io/projected/1e75af59-3cca-45fc-babf-990d767c7467-kube-api-access-ghhmg\") pod \"maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.496289 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.496128 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e75af59-3cca-45fc-babf-990d767c7467-openshift-service-ca\") pod \"maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.496289 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.496201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e75af59-3cca-45fc-babf-990d767c7467-maas-api-tls\") pod \"maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.597319 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.597283 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e75af59-3cca-45fc-babf-990d767c7467-openshift-service-ca\") pod \"maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.597514 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.597366 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e75af59-3cca-45fc-babf-990d767c7467-maas-api-tls\") pod \"maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.597891 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.597856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghhmg\" (UniqueName: \"kubernetes.io/projected/1e75af59-3cca-45fc-babf-990d767c7467-kube-api-access-ghhmg\") pod \"maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.598125 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.598078 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e75af59-3cca-45fc-babf-990d767c7467-openshift-service-ca\") pod \"maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.599879 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.599857 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e75af59-3cca-45fc-babf-990d767c7467-maas-api-tls\") pod \"maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.609839 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.609813 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghhmg\" (UniqueName: \"kubernetes.io/projected/1e75af59-3cca-45fc-babf-990d767c7467-kube-api-access-ghhmg\") pod \"maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.705421 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.705380 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:43.866161 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:33:43.866130 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e75af59_3cca_45fc_babf_990d767c7467.slice/crio-825377dd0984f4cf305358f43a3f18f967709e167bc147ce3c44e8922495e10c WatchSource:0}: Error finding container 825377dd0984f4cf305358f43a3f18f967709e167bc147ce3c44e8922495e10c: Status 404 returned error can't find the container with id 825377dd0984f4cf305358f43a3f18f967709e167bc147ce3c44e8922495e10c Jul 03 10:33:43.868440 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:43.868324 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb"] Jul 03 10:33:44.117260 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:44.117139 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" event={"ID":"1e75af59-3cca-45fc-babf-990d767c7467","Type":"ContainerStarted","Data":"825377dd0984f4cf305358f43a3f18f967709e167bc147ce3c44e8922495e10c"} Jul 03 10:33:45.124015 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:45.123974 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" event={"ID":"1e75af59-3cca-45fc-babf-990d767c7467","Type":"ContainerStarted","Data":"7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3"} Jul 03 10:33:45.124474 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:45.124230 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:45.152148 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:45.152072 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" podStartSLOduration=1.709468394 podStartE2EDuration="2.152054309s" totalImagesPullingTime="442.585915ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:33:45.150356372 +0000 UTC m=+1703.216431578" watchObservedRunningTime="2026-07-03 10:33:45.152054309 +0000 UTC m=+1703.218129501" Jul 03 10:33:51.141882 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:51.141841 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:55.431029 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.430988 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb"] Jul 03 10:33:55.431876 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.431816 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" podUID="1e75af59-3cca-45fc-babf-990d767c7467" containerName="maas-api" containerID="cri-o://7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3" gracePeriod=30 Jul 03 10:33:55.687856 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.687769 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:55.815398 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.815357 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e75af59-3cca-45fc-babf-990d767c7467-openshift-service-ca\") pod \"1e75af59-3cca-45fc-babf-990d767c7467\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " Jul 03 10:33:55.815398 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.815404 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghhmg\" (UniqueName: \"kubernetes.io/projected/1e75af59-3cca-45fc-babf-990d767c7467-kube-api-access-ghhmg\") pod \"1e75af59-3cca-45fc-babf-990d767c7467\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " Jul 03 10:33:55.815628 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.815425 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e75af59-3cca-45fc-babf-990d767c7467-maas-api-tls\") pod \"1e75af59-3cca-45fc-babf-990d767c7467\" (UID: \"1e75af59-3cca-45fc-babf-990d767c7467\") " Jul 03 10:33:55.815749 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.815725 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e75af59-3cca-45fc-babf-990d767c7467-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1e75af59-3cca-45fc-babf-990d767c7467" (UID: "1e75af59-3cca-45fc-babf-990d767c7467"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:33:55.817589 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.817552 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e75af59-3cca-45fc-babf-990d767c7467-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1e75af59-3cca-45fc-babf-990d767c7467" (UID: "1e75af59-3cca-45fc-babf-990d767c7467"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:33:55.817589 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.817580 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e75af59-3cca-45fc-babf-990d767c7467-kube-api-access-ghhmg" (OuterVolumeSpecName: "kube-api-access-ghhmg") pod "1e75af59-3cca-45fc-babf-990d767c7467" (UID: "1e75af59-3cca-45fc-babf-990d767c7467"). InnerVolumeSpecName "kube-api-access-ghhmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:33:55.916231 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.916192 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e75af59-3cca-45fc-babf-990d767c7467-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:33:55.916231 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.916223 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghhmg\" (UniqueName: \"kubernetes.io/projected/1e75af59-3cca-45fc-babf-990d767c7467-kube-api-access-ghhmg\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:33:55.916231 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:55.916232 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e75af59-3cca-45fc-babf-990d767c7467-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:33:56.190816 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.190784 2577 generic.go:356] "Generic (PLEG): container finished" podID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerID="e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed" exitCode=0 Jul 03 10:33:56.191043 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.190867 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" event={"ID":"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43","Type":"ContainerDied","Data":"e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed"} Jul 03 10:33:56.191406 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.191384 2577 scope.go:117] "RemoveContainer" containerID="e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed" Jul 03 10:33:56.192188 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.192165 2577 generic.go:356] "Generic (PLEG): container finished" podID="1e75af59-3cca-45fc-babf-990d767c7467" containerID="7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3" exitCode=0 Jul 03 10:33:56.192302 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.192203 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" event={"ID":"1e75af59-3cca-45fc-babf-990d767c7467","Type":"ContainerDied","Data":"7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3"} Jul 03 10:33:56.192302 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.192232 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" Jul 03 10:33:56.192302 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.192256 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb" event={"ID":"1e75af59-3cca-45fc-babf-990d767c7467","Type":"ContainerDied","Data":"825377dd0984f4cf305358f43a3f18f967709e167bc147ce3c44e8922495e10c"} Jul 03 10:33:56.192302 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.192287 2577 scope.go:117] "RemoveContainer" containerID="7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3" Jul 03 10:33:56.192464 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.192431 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:33:56.192532 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.192509 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:33:56.192579 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.192555 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:33:56.206450 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.206406 2577 scope.go:117] "RemoveContainer" containerID="7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3" Jul 03 10:33:56.206763 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:33:56.206745 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3\": container with ID starting with 7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3 not found: ID does not exist" containerID="7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3" Jul 03 10:33:56.206822 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.206772 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3"} err="failed to get container status \"7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3\": rpc error: code = NotFound desc = could not find container \"7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3\": container with ID starting with 7f6fe21afdb1de3567e90db96deccfb93983f65797177ca213adbd8c25d6eda3 not found: ID does not exist" Jul 03 10:33:56.242795 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.242754 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb"] Jul 03 10:33:56.251366 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.251338 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-0d436a63-5895c6bd8b-7r5tb"] Jul 03 10:33:56.473149 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:56.473113 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e75af59-3cca-45fc-babf-990d767c7467" path="/var/lib/kubelet/pods/1e75af59-3cca-45fc-babf-990d767c7467/volumes" Jul 03 10:33:57.104186 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:57.104142 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:57.199112 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:57.199063 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" event={"ID":"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43","Type":"ContainerStarted","Data":"98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0"} Jul 03 10:33:57.527622 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:57.527589 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:33:57.528782 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:57.528756 2577 patch_prober.go:28] interesting pod/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 03 10:33:57.528871 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:57.528806 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 03 10:33:58.527984 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:58.527938 2577 patch_prober.go:28] interesting pod/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 03 10:33:58.528449 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:58.527997 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 03 10:33:59.527338 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:59.527298 2577 patch_prober.go:28] interesting pod/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 03 10:33:59.527531 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:33:59.527352 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 03 10:34:00.528427 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:00.528383 2577 patch_prober.go:28] interesting pod/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 03 10:34:00.528847 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:00.528453 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 03 10:34:01.531424 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:01.531387 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:34:01.531885 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:01.531670 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:34:01.532346 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:01.532329 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:34:02.056074 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:02.056034 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr"] Jul 03 10:34:02.225144 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:02.225082 2577 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" secret="" err="secret \"e2e-mt-0d436a63-openshift-default-dockercfg-7fvn9\" not found" Jul 03 10:34:03.229444 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:03.229398 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" containerID="cri-o://98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0" gracePeriod=30 Jul 03 10:34:08.487874 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.487847 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:34:08.536648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.536549 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istiod-ca-cert\") pod \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " Jul 03 10:34:08.536648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.536592 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-token\") pod \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " Jul 03 10:34:08.536648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.536611 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-data\") pod \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " Jul 03 10:34:08.536949 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.536661 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-envoy\") pod \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " Jul 03 10:34:08.536949 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.536680 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-socket\") pod \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " Jul 03 10:34:08.536949 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.536700 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-credential-socket\") pod \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " Jul 03 10:34:08.536949 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.536753 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jndh2\" (UniqueName: \"kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-kube-api-access-jndh2\") pod \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " Jul 03 10:34:08.536949 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.536780 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-podinfo\") pod \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " Jul 03 10:34:08.536949 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.536804 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-certs\") pod \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\" (UID: \"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43\") " Jul 03 10:34:08.537302 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.537272 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" (UID: "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:34:08.537362 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.537276 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" (UID: "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:34:08.537416 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.537373 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" (UID: "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:34:08.537588 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.537557 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" (UID: "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:34:08.537720 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.537599 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-data" (OuterVolumeSpecName: "istio-data") pod "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" (UID: "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:34:08.539042 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.539003 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" (UID: "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:34:08.539316 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.539295 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-token" (OuterVolumeSpecName: "istio-token") pod "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" (UID: "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:34:08.539430 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.539407 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-kube-api-access-jndh2" (OuterVolumeSpecName: "kube-api-access-jndh2") pod "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" (UID: "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43"). InnerVolumeSpecName "kube-api-access-jndh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:34:08.539475 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.539416 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" (UID: "cb098c77-87b0-462d-a5b7-f5e7dcd5ce43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:34:08.637900 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.637864 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:08.637900 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.637891 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:08.637900 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.637899 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:08.637900 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.637910 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jndh2\" (UniqueName: \"kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-kube-api-access-jndh2\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:08.638203 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.637920 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:08.638203 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.637929 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:08.638203 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.637937 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:08.638203 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.637944 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:08.638203 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:08.637951 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:09.260198 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.260159 2577 generic.go:356] "Generic (PLEG): container finished" podID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerID="98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0" exitCode=0 Jul 03 10:34:09.260405 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.260195 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" event={"ID":"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43","Type":"ContainerDied","Data":"98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0"} Jul 03 10:34:09.260405 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.260218 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" Jul 03 10:34:09.260405 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.260257 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr" event={"ID":"cb098c77-87b0-462d-a5b7-f5e7dcd5ce43","Type":"ContainerDied","Data":"a616d6b8f67808990d28c5eb318a2a1dcdec90aa7b4c9674b75105001b52e73d"} Jul 03 10:34:09.260405 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.260283 2577 scope.go:117] "RemoveContainer" containerID="98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0" Jul 03 10:34:09.269704 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.269688 2577 scope.go:117] "RemoveContainer" containerID="e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed" Jul 03 10:34:09.277911 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.277893 2577 scope.go:117] "RemoveContainer" containerID="98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0" Jul 03 10:34:09.278191 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:34:09.278170 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0\": container with ID starting with 98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0 not found: ID does not exist" containerID="98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0" Jul 03 10:34:09.278238 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.278203 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0"} err="failed to get container status \"98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0\": rpc error: code = NotFound desc = could not find container \"98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0\": container with ID starting with 98dac3a187fe6a1398b069a9de541f2cd36ddeacc94e76e0ef98b05dcb5860f0 not found: ID does not exist" Jul 03 10:34:09.278238 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.278225 2577 scope.go:117] "RemoveContainer" containerID="e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed" Jul 03 10:34:09.278467 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:34:09.278447 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed\": container with ID starting with e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed not found: ID does not exist" containerID="e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed" Jul 03 10:34:09.278508 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.278474 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed"} err="failed to get container status \"e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed\": rpc error: code = NotFound desc = could not find container \"e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed\": container with ID starting with e1f672841c25679afb3840482d89014d924247303b1dfbc6cb9781fedf5884ed not found: ID does not exist" Jul 03 10:34:09.284151 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.284129 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr"] Jul 03 10:34:09.288658 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:09.288637 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0d436a63-openshift-default-55478fdcc5-cdzkr"] Jul 03 10:34:10.471515 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:10.471482 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" path="/var/lib/kubelet/pods/cb098c77-87b0-462d-a5b7-f5e7dcd5ce43/volumes" Jul 03 10:34:21.666757 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.666718 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb"] Jul 03 10:34:21.667250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.667060 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e75af59-3cca-45fc-babf-990d767c7467" containerName="maas-api" Jul 03 10:34:21.667250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.667071 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e75af59-3cca-45fc-babf-990d767c7467" containerName="maas-api" Jul 03 10:34:21.667250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.667081 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" Jul 03 10:34:21.667250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.667087 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" Jul 03 10:34:21.667250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.667111 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" Jul 03 10:34:21.667250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.667117 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" Jul 03 10:34:21.667250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.667182 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" Jul 03 10:34:21.667250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.667190 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e75af59-3cca-45fc-babf-990d767c7467" containerName="maas-api" Jul 03 10:34:21.667250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.667198 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb098c77-87b0-462d-a5b7-f5e7dcd5ce43" containerName="istio-proxy" Jul 03 10:34:21.669871 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.669854 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.675709 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.675681 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-dcde78d2-openshift-default-dockercfg-hlfd2\"" Jul 03 10:34:21.690078 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.690048 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb"] Jul 03 10:34:21.763995 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.763952 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-envoy\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.764216 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.763999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-credential-socket\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.764216 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.764027 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8wwh\" (UniqueName: \"kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-kube-api-access-z8wwh\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.764216 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.764084 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-podinfo\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.764216 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.764164 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-token\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.764414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.764223 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-data\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.764414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.764280 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istiod-ca-cert\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.764414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.764333 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-socket\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.764414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.764359 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-certs\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865256 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-data\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865256 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istiod-ca-cert\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865536 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865296 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-socket\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865536 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865319 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-certs\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865536 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865347 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-envoy\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865536 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865375 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-credential-socket\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865536 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865400 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8wwh\" (UniqueName: \"kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-kube-api-access-z8wwh\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865536 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-podinfo\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865536 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-token\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865887 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865670 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-data\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865887 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865717 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-socket\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.865887 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865752 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-certs\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.866002 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.865898 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-credential-socket\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.866247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.866223 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istiod-ca-cert\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.867938 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.867918 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-envoy\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.868061 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.868045 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-podinfo\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.875406 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.875379 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-token\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.875638 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.875616 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8wwh\" (UniqueName: \"kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-kube-api-access-z8wwh\") pod \"e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:21.979733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:21.979647 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:22.129764 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:22.129733 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb"] Jul 03 10:34:22.131343 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:34:22.131311 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9621f2_2f74_4aa7_a642_67a7bab3ea9d.slice/crio-181d5459f2663d52d528207ca42bc5273de465ae0e824e2fc764365abf884ef0 WatchSource:0}: Error finding container 181d5459f2663d52d528207ca42bc5273de465ae0e824e2fc764365abf884ef0: Status 404 returned error can't find the container with id 181d5459f2663d52d528207ca42bc5273de465ae0e824e2fc764365abf884ef0 Jul 03 10:34:22.133520 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:22.133480 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:34:22.133609 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:22.133571 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:34:22.133649 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:22.133615 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:34:22.330876 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:22.330770 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" event={"ID":"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d","Type":"ContainerStarted","Data":"eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3"} Jul 03 10:34:22.330876 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:22.330827 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" event={"ID":"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d","Type":"ContainerStarted","Data":"181d5459f2663d52d528207ca42bc5273de465ae0e824e2fc764365abf884ef0"} Jul 03 10:34:22.359374 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:22.359324 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" podStartSLOduration=1.359307877 podStartE2EDuration="1.359307877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:34:22.355544022 +0000 UTC m=+1740.421619214" watchObservedRunningTime="2026-07-03 10:34:22.359307877 +0000 UTC m=+1740.425383142" Jul 03 10:34:22.980523 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:22.980486 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:23.985246 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:23.985211 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:24.341299 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.341218 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:24.342189 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.342170 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:24.574819 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.574787 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4"] Jul 03 10:34:24.577543 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.577524 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:24.581307 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.581284 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-dcde78d2-serving-cert\"" Jul 03 10:34:24.590875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.590850 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4"] Jul 03 10:34:24.692006 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.691966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42e8fe50-acdd-4b35-b702-42e75b367dab-maas-api-tls\") pod \"maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:24.692259 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.692018 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz4dq\" (UniqueName: \"kubernetes.io/projected/42e8fe50-acdd-4b35-b702-42e75b367dab-kube-api-access-nz4dq\") pod \"maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:24.692259 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.692077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42e8fe50-acdd-4b35-b702-42e75b367dab-openshift-service-ca\") pod \"maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:24.793254 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.793214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42e8fe50-acdd-4b35-b702-42e75b367dab-maas-api-tls\") pod \"maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:24.793471 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.793259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz4dq\" (UniqueName: \"kubernetes.io/projected/42e8fe50-acdd-4b35-b702-42e75b367dab-kube-api-access-nz4dq\") pod \"maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:24.793471 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.793305 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42e8fe50-acdd-4b35-b702-42e75b367dab-openshift-service-ca\") pod \"maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:24.793471 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:34:24.793323 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-dcde78d2-serving-cert: secret "maas-api-e2e-mt-dcde78d2-serving-cert" not found Jul 03 10:34:24.793471 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:34:24.793384 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e8fe50-acdd-4b35-b702-42e75b367dab-maas-api-tls podName:42e8fe50-acdd-4b35-b702-42e75b367dab nodeName:}" failed. No retries permitted until 2026-07-03 10:34:25.293367475 +0000 UTC m=+1743.359442649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/42e8fe50-acdd-4b35-b702-42e75b367dab-maas-api-tls") pod "maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" (UID: "42e8fe50-acdd-4b35-b702-42e75b367dab") : secret "maas-api-e2e-mt-dcde78d2-serving-cert" not found Jul 03 10:34:24.794010 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.793987 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42e8fe50-acdd-4b35-b702-42e75b367dab-openshift-service-ca\") pod \"maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:24.805335 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:24.805308 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz4dq\" (UniqueName: \"kubernetes.io/projected/42e8fe50-acdd-4b35-b702-42e75b367dab-kube-api-access-nz4dq\") pod \"maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:25.297815 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:25.297777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42e8fe50-acdd-4b35-b702-42e75b367dab-maas-api-tls\") pod \"maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:25.300393 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:25.300367 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42e8fe50-acdd-4b35-b702-42e75b367dab-maas-api-tls\") pod \"maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:25.488540 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:25.488500 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:25.628463 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:25.628435 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4"] Jul 03 10:34:25.630538 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:34:25.630510 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e8fe50_acdd_4b35_b702_42e75b367dab.slice/crio-19dc3063fa026b83484eea3324fb67a0e2373b5c41fcc586bd34b3b17acf51fb WatchSource:0}: Error finding container 19dc3063fa026b83484eea3324fb67a0e2373b5c41fcc586bd34b3b17acf51fb: Status 404 returned error can't find the container with id 19dc3063fa026b83484eea3324fb67a0e2373b5c41fcc586bd34b3b17acf51fb Jul 03 10:34:26.353026 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:26.352930 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" event={"ID":"42e8fe50-acdd-4b35-b702-42e75b367dab","Type":"ContainerStarted","Data":"915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933"} Jul 03 10:34:26.353469 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:26.353207 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:26.353469 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:26.353241 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" event={"ID":"42e8fe50-acdd-4b35-b702-42e75b367dab","Type":"ContainerStarted","Data":"19dc3063fa026b83484eea3324fb67a0e2373b5c41fcc586bd34b3b17acf51fb"} Jul 03 10:34:26.373384 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:26.373327 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" podStartSLOduration=2.016071967 podStartE2EDuration="2.373312269s" totalImagesPullingTime="357.240302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:34:26.370051528 +0000 UTC m=+1744.436126720" watchObservedRunningTime="2026-07-03 10:34:26.373312269 +0000 UTC m=+1744.439387461" Jul 03 10:34:32.364671 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:32.364643 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:35.400535 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:35.400501 2577 generic.go:356] "Generic (PLEG): container finished" podID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerID="eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3" exitCode=0 Jul 03 10:34:35.401001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:35.400576 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" event={"ID":"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d","Type":"ContainerDied","Data":"eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3"} Jul 03 10:34:35.401001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:35.400965 2577 scope.go:117] "RemoveContainer" containerID="eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3" Jul 03 10:34:35.402002 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:35.401975 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:34:35.402063 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:35.402046 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:34:35.402126 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:35.402088 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:34:36.408957 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:36.408919 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" event={"ID":"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d","Type":"ContainerStarted","Data":"dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237"} Jul 03 10:34:36.980881 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:36.980838 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:36.982014 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:36.981985 2577 patch_prober.go:28] interesting pod/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 03 10:34:36.982157 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:36.982049 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 03 10:34:37.980894 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:37.980851 2577 patch_prober.go:28] interesting pod/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 03 10:34:37.981355 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:37.980921 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 03 10:34:38.980654 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:38.980614 2577 patch_prober.go:28] interesting pod/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 03 10:34:38.980847 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:38.980672 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 03 10:34:39.341874 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:39.341762 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:39.980498 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:39.980463 2577 patch_prober.go:28] interesting pod/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 03 10:34:39.980697 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:39.980520 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 03 10:34:40.985167 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:40.985135 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:40.986136 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:40.986116 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:34:56.825896 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:56.825847 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4"] Jul 03 10:34:56.827065 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:56.826776 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" podUID="42e8fe50-acdd-4b35-b702-42e75b367dab" containerName="maas-api" containerID="cri-o://915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933" gracePeriod=30 Jul 03 10:34:57.093289 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.093262 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:57.193965 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.193928 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz4dq\" (UniqueName: \"kubernetes.io/projected/42e8fe50-acdd-4b35-b702-42e75b367dab-kube-api-access-nz4dq\") pod \"42e8fe50-acdd-4b35-b702-42e75b367dab\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " Jul 03 10:34:57.194211 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.194001 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42e8fe50-acdd-4b35-b702-42e75b367dab-openshift-service-ca\") pod \"42e8fe50-acdd-4b35-b702-42e75b367dab\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " Jul 03 10:34:57.194211 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.194049 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42e8fe50-acdd-4b35-b702-42e75b367dab-maas-api-tls\") pod \"42e8fe50-acdd-4b35-b702-42e75b367dab\" (UID: \"42e8fe50-acdd-4b35-b702-42e75b367dab\") " Jul 03 10:34:57.194417 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.194390 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42e8fe50-acdd-4b35-b702-42e75b367dab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42e8fe50-acdd-4b35-b702-42e75b367dab" (UID: "42e8fe50-acdd-4b35-b702-42e75b367dab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:34:57.196171 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.196145 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42e8fe50-acdd-4b35-b702-42e75b367dab-kube-api-access-nz4dq" (OuterVolumeSpecName: "kube-api-access-nz4dq") pod "42e8fe50-acdd-4b35-b702-42e75b367dab" (UID: "42e8fe50-acdd-4b35-b702-42e75b367dab"). InnerVolumeSpecName "kube-api-access-nz4dq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:34:57.196288 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.196153 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42e8fe50-acdd-4b35-b702-42e75b367dab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42e8fe50-acdd-4b35-b702-42e75b367dab" (UID: "42e8fe50-acdd-4b35-b702-42e75b367dab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:34:57.295717 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.295679 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42e8fe50-acdd-4b35-b702-42e75b367dab-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:57.295717 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.295711 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42e8fe50-acdd-4b35-b702-42e75b367dab-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:57.295717 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.295720 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz4dq\" (UniqueName: \"kubernetes.io/projected/42e8fe50-acdd-4b35-b702-42e75b367dab-kube-api-access-nz4dq\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:34:57.518904 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.518872 2577 generic.go:356] "Generic (PLEG): container finished" podID="42e8fe50-acdd-4b35-b702-42e75b367dab" containerID="915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933" exitCode=0 Jul 03 10:34:57.519114 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.518929 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" Jul 03 10:34:57.519114 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.518935 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" event={"ID":"42e8fe50-acdd-4b35-b702-42e75b367dab","Type":"ContainerDied","Data":"915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933"} Jul 03 10:34:57.519114 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.518992 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4" event={"ID":"42e8fe50-acdd-4b35-b702-42e75b367dab","Type":"ContainerDied","Data":"19dc3063fa026b83484eea3324fb67a0e2373b5c41fcc586bd34b3b17acf51fb"} Jul 03 10:34:57.519114 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.519015 2577 scope.go:117] "RemoveContainer" containerID="915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933" Jul 03 10:34:57.528537 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.528516 2577 scope.go:117] "RemoveContainer" containerID="915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933" Jul 03 10:34:57.528827 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:34:57.528810 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933\": container with ID starting with 915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933 not found: ID does not exist" containerID="915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933" Jul 03 10:34:57.528899 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.528838 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933"} err="failed to get container status \"915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933\": rpc error: code = NotFound desc = could not find container \"915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933\": container with ID starting with 915d2f2126d7be888f2108a5d7ac2e10e10e4bdaceb851d320d7a06d31d90933 not found: ID does not exist" Jul 03 10:34:57.546522 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.546474 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4"] Jul 03 10:34:57.553141 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:57.553115 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-dcde78d2-67b5bd995d-v7kb4"] Jul 03 10:34:58.477647 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:34:58.477609 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42e8fe50-acdd-4b35-b702-42e75b367dab" path="/var/lib/kubelet/pods/42e8fe50-acdd-4b35-b702-42e75b367dab/volumes" Jul 03 10:35:22.508502 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:35:22.508470 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:35:22.512699 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:35:22.512678 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:35:22.513973 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:35:22.513956 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:35:22.518344 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:35:22.518324 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:35:56.473804 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:35:56.473768 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb"] Jul 03 10:35:56.474351 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:35:56.474020 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" containerID="cri-o://dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237" gracePeriod=30 Jul 03 10:36:01.735163 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.735078 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:36:01.794990 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.794956 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-certs\") pod \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " Jul 03 10:36:01.795223 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795015 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-data\") pod \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " Jul 03 10:36:01.795223 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795043 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-envoy\") pod \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " Jul 03 10:36:01.795223 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795087 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istiod-ca-cert\") pod \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " Jul 03 10:36:01.795223 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795169 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-socket\") pod \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " Jul 03 10:36:01.795456 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795231 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-podinfo\") pod \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " Jul 03 10:36:01.795456 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795260 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-token\") pod \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " Jul 03 10:36:01.795456 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795291 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-credential-socket\") pod \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " Jul 03 10:36:01.795456 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795298 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" (UID: "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:36:01.795456 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795323 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8wwh\" (UniqueName: \"kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-kube-api-access-z8wwh\") pod \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\" (UID: \"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d\") " Jul 03 10:36:01.795732 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795529 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" (UID: "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:36:01.795732 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795704 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:36:01.795732 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795725 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:36:01.795898 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795784 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" (UID: "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:36:01.795966 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795915 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-data" (OuterVolumeSpecName: "istio-data") pod "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" (UID: "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:36:01.796019 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.795969 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" (UID: "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:36:01.797459 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.797423 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" (UID: "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:36:01.797641 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.797617 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-token" (OuterVolumeSpecName: "istio-token") pod "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" (UID: "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:36:01.797775 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.797754 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" (UID: "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:36:01.797841 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.797792 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-kube-api-access-z8wwh" (OuterVolumeSpecName: "kube-api-access-z8wwh") pod "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" (UID: "8d9621f2-2f74-4aa7-a642-67a7bab3ea9d"). InnerVolumeSpecName "kube-api-access-z8wwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:36:01.863133 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.863083 2577 generic.go:356] "Generic (PLEG): container finished" podID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerID="dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237" exitCode=0 Jul 03 10:36:01.863304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.863157 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" Jul 03 10:36:01.863304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.863169 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" event={"ID":"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d","Type":"ContainerDied","Data":"dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237"} Jul 03 10:36:01.863304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.863225 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb" event={"ID":"8d9621f2-2f74-4aa7-a642-67a7bab3ea9d","Type":"ContainerDied","Data":"181d5459f2663d52d528207ca42bc5273de465ae0e824e2fc764365abf884ef0"} Jul 03 10:36:01.863304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.863247 2577 scope.go:117] "RemoveContainer" containerID="dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237" Jul 03 10:36:01.873014 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.872993 2577 scope.go:117] "RemoveContainer" containerID="eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3" Jul 03 10:36:01.882071 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.882050 2577 scope.go:117] "RemoveContainer" containerID="dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237" Jul 03 10:36:01.882384 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:36:01.882362 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237\": container with ID starting with dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237 not found: ID does not exist" containerID="dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237" Jul 03 10:36:01.882473 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.882398 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237"} err="failed to get container status \"dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237\": rpc error: code = NotFound desc = could not find container \"dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237\": container with ID starting with dd8d3b3a8c243d5d40fa2f4c254b8ceffd3326e9e95441f92d57370793dd9237 not found: ID does not exist" Jul 03 10:36:01.882473 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.882423 2577 scope.go:117] "RemoveContainer" containerID="eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3" Jul 03 10:36:01.882705 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:36:01.882687 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3\": container with ID starting with eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3 not found: ID does not exist" containerID="eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3" Jul 03 10:36:01.882746 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.882712 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3"} err="failed to get container status \"eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3\": rpc error: code = NotFound desc = could not find container \"eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3\": container with ID starting with eeba5738acebda0609df44519ea2adfde285639ef374bbd8455a8fe3363e78d3 not found: ID does not exist" Jul 03 10:36:01.895046 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.895015 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb"] Jul 03 10:36:01.896885 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.896867 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:36:01.896941 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.896888 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:36:01.896941 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.896898 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:36:01.896941 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.896907 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8wwh\" (UniqueName: \"kubernetes.io/projected/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-kube-api-access-z8wwh\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:36:01.896941 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.896917 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:36:01.896941 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.896926 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:36:01.896941 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.896935 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:36:01.903036 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:01.903006 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-dcde78d2-openshift-default-6bc8745b99-k55sb"] Jul 03 10:36:02.472367 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:02.472336 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" path="/var/lib/kubelet/pods/8d9621f2-2f74-4aa7-a642-67a7bab3ea9d/volumes" Jul 03 10:36:52.022039 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.021955 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68"] Jul 03 10:36:52.022532 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.022340 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" Jul 03 10:36:52.022532 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.022351 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" Jul 03 10:36:52.022532 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.022361 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e8fe50-acdd-4b35-b702-42e75b367dab" containerName="maas-api" Jul 03 10:36:52.022532 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.022366 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e8fe50-acdd-4b35-b702-42e75b367dab" containerName="maas-api" Jul 03 10:36:52.022532 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.022430 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="42e8fe50-acdd-4b35-b702-42e75b367dab" containerName="maas-api" Jul 03 10:36:52.022532 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.022439 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" Jul 03 10:36:52.022532 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.022446 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" Jul 03 10:36:52.025552 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.025531 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.028877 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.028849 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1d628f-openshift-default-dockercfg-hq4t6\"" Jul 03 10:36:52.043502 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.043464 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68"] Jul 03 10:36:52.156471 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.156435 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-token\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.156668 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.156480 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32ec802-99b7-44ab-aebe-c42b5572d653-istiod-ca-cert\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.156668 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.156505 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-socket\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.156668 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.156542 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-data\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.156668 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.156569 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4226l\" (UniqueName: \"kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-kube-api-access-4226l\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.156668 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.156587 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-credential-socket\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.156668 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.156641 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-certs\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.156921 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.156687 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-envoy\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.156921 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.156787 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-podinfo\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.257490 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257451 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32ec802-99b7-44ab-aebe-c42b5572d653-istiod-ca-cert\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.257490 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257489 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-socket\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.257751 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257506 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-data\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.257751 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257624 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4226l\" (UniqueName: \"kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-kube-api-access-4226l\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.257751 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257656 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-credential-socket\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.257913 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257789 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-certs\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.257913 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257823 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-envoy\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.258020 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257910 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-data\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.258020 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257914 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-podinfo\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.258020 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257977 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-credential-socket\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.258020 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.257936 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-socket\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.258249 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.258038 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-token\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.258249 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.258211 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-certs\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.258546 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.258526 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32ec802-99b7-44ab-aebe-c42b5572d653-istiod-ca-cert\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.260225 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.260203 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-envoy\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.260429 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.260413 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-podinfo\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.268420 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.268387 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4226l\" (UniqueName: \"kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-kube-api-access-4226l\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.269888 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.269869 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-token\") pod \"e2e-inf-a-1d628f-openshift-default-848678f755-5jg68\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.335445 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.335350 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:52.490052 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.490023 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68"] Jul 03 10:36:52.492067 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:36:52.492044 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda32ec802_99b7_44ab_aebe_c42b5572d653.slice/crio-442c92bf3f7301503d2ad58330e6c485ff5a89400afd0b96241ecd8bd4c95bb6 WatchSource:0}: Error finding container 442c92bf3f7301503d2ad58330e6c485ff5a89400afd0b96241ecd8bd4c95bb6: Status 404 returned error can't find the container with id 442c92bf3f7301503d2ad58330e6c485ff5a89400afd0b96241ecd8bd4c95bb6 Jul 03 10:36:52.494214 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.494181 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:36:52.494291 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.494245 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:36:52.494291 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:52.494274 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:36:53.135002 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:53.134965 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" event={"ID":"a32ec802-99b7-44ab-aebe-c42b5572d653","Type":"ContainerStarted","Data":"4bdfc47405bcd5e95726d8afade3f87bae9f4505624b7bdc87b5dd6bf2d21c37"} Jul 03 10:36:53.135447 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:53.135019 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" event={"ID":"a32ec802-99b7-44ab-aebe-c42b5572d653","Type":"ContainerStarted","Data":"442c92bf3f7301503d2ad58330e6c485ff5a89400afd0b96241ecd8bd4c95bb6"} Jul 03 10:36:53.168164 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:53.168107 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" podStartSLOduration=2.168072852 podStartE2EDuration="2.168072852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:36:53.166548471 +0000 UTC m=+1891.232623663" watchObservedRunningTime="2026-07-03 10:36:53.168072852 +0000 UTC m=+1891.234148043" Jul 03 10:36:53.336021 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:53.335978 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:53.341054 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:53.341027 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:54.140520 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:54.140488 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:54.141494 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:54.141475 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:36:55.336875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.336836 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc"] Jul 03 10:36:55.337414 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.337393 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" Jul 03 10:36:55.337511 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.337417 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d9621f2-2f74-4aa7-a642-67a7bab3ea9d" containerName="istio-proxy" Jul 03 10:36:55.340471 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.340451 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:55.343801 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.343777 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-1d628f-serving-cert\"" Jul 03 10:36:55.365016 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.364972 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc"] Jul 03 10:36:55.488479 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.488439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:55.488479 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.488479 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj9r5\" (UniqueName: \"kubernetes.io/projected/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-kube-api-access-fj9r5\") pod \"maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:55.488707 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.488553 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:55.589367 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.589275 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:55.589545 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.589382 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:55.589545 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.589406 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj9r5\" (UniqueName: \"kubernetes.io/projected/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-kube-api-access-fj9r5\") pod \"maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:55.589545 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:36:55.589537 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-1d628f-serving-cert: secret "maas-api-e2e-inf-a-1d628f-serving-cert" not found Jul 03 10:36:55.589719 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:36:55.589610 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-maas-api-tls podName:0de1a249-9a0c-467a-8efd-fc01b9b8d9bd nodeName:}" failed. No retries permitted until 2026-07-03 10:36:56.089591473 +0000 UTC m=+1894.155666658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-maas-api-tls") pod "maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" (UID: "0de1a249-9a0c-467a-8efd-fc01b9b8d9bd") : secret "maas-api-e2e-inf-a-1d628f-serving-cert" not found Jul 03 10:36:55.589970 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.589949 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:55.624853 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:55.624813 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj9r5\" (UniqueName: \"kubernetes.io/projected/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-kube-api-access-fj9r5\") pod \"maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:56.095580 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:56.095534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:56.097996 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:56.097970 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:56.251835 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:56.251802 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:56.399728 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:56.399694 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc"] Jul 03 10:36:56.400879 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:36:56.400851 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de1a249_9a0c_467a_8efd_fc01b9b8d9bd.slice/crio-3f9d90ec79b7617d7bdb6c4f52d9817ea3dda11c23dfc179899eff8892985e12 WatchSource:0}: Error finding container 3f9d90ec79b7617d7bdb6c4f52d9817ea3dda11c23dfc179899eff8892985e12: Status 404 returned error can't find the container with id 3f9d90ec79b7617d7bdb6c4f52d9817ea3dda11c23dfc179899eff8892985e12 Jul 03 10:36:56.402321 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:56.402302 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:36:57.157559 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:57.157526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" event={"ID":"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd","Type":"ContainerStarted","Data":"c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3"} Jul 03 10:36:57.157748 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:57.157579 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" event={"ID":"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd","Type":"ContainerStarted","Data":"3f9d90ec79b7617d7bdb6c4f52d9817ea3dda11c23dfc179899eff8892985e12"} Jul 03 10:36:57.157748 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:57.157676 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:36:57.185862 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:36:57.185800 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" podStartSLOduration=1.779120565 podStartE2EDuration="2.185783096s" totalImagesPullingTime="406.662531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:36:57.184237592 +0000 UTC m=+1895.250312797" watchObservedRunningTime="2026-07-03 10:36:57.185783096 +0000 UTC m=+1895.251858287" Jul 03 10:37:00.477511 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.477476 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7"] Jul 03 10:37:00.481245 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.481228 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.484696 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.484671 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7436d1-openshift-default-dockercfg-mrls6\"" Jul 03 10:37:00.511441 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.511405 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7"] Jul 03 10:37:00.636507 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.636471 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-token\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.636689 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.636507 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istiod-ca-cert\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.636689 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.636577 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-data\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.636689 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.636600 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-envoy\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.636689 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.636619 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-podinfo\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.636689 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.636658 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-credential-socket\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.636860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.636755 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-socket\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.636860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.636839 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-certs\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.636927 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.636859 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sxmt\" (UniqueName: \"kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-kube-api-access-2sxmt\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.737640 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.737545 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-data\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.737640 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.737588 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-envoy\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.737640 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.737617 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-podinfo\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.737923 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.737651 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-credential-socket\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.737923 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.737678 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-socket\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.737923 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.737760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-certs\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.737923 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.737789 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxmt\" (UniqueName: \"kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-kube-api-access-2sxmt\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.737923 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.737820 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-token\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.737923 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.737842 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istiod-ca-cert\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.738264 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.738015 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-data\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.738264 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.738203 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-credential-socket\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.738457 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.738436 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-socket\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.738585 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.738566 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-certs\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.738633 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.738600 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istiod-ca-cert\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.740280 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.740243 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-envoy\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.740559 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.740538 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-podinfo\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.761819 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.761785 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-token\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.761958 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.761931 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sxmt\" (UniqueName: \"kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-kube-api-access-2sxmt\") pod \"e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.790930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.790894 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:00.976536 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.976501 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7"] Jul 03 10:37:00.984434 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:37:00.984361 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73fad2f_56d6_4bb6_8874_587bc39bc91d.slice/crio-6897ac1b2c3812e030f0e5c7a6ac7cd9b14bf65f0bb8b96ed46eb8774fafce32 WatchSource:0}: Error finding container 6897ac1b2c3812e030f0e5c7a6ac7cd9b14bf65f0bb8b96ed46eb8774fafce32: Status 404 returned error can't find the container with id 6897ac1b2c3812e030f0e5c7a6ac7cd9b14bf65f0bb8b96ed46eb8774fafce32 Jul 03 10:37:00.986736 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.986701 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:37:00.986836 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.986786 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:37:00.986880 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:00.986833 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:37:01.180809 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:01.180769 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" event={"ID":"c73fad2f-56d6-4bb6-8874-587bc39bc91d","Type":"ContainerStarted","Data":"8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1"} Jul 03 10:37:01.180991 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:01.180836 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" event={"ID":"c73fad2f-56d6-4bb6-8874-587bc39bc91d","Type":"ContainerStarted","Data":"6897ac1b2c3812e030f0e5c7a6ac7cd9b14bf65f0bb8b96ed46eb8774fafce32"} Jul 03 10:37:01.234806 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:01.234740 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" podStartSLOduration=1.234716358 podStartE2EDuration="1.234716358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:37:01.233659785 +0000 UTC m=+1899.299734980" watchObservedRunningTime="2026-07-03 10:37:01.234716358 +0000 UTC m=+1899.300791551" Jul 03 10:37:01.791034 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:01.790990 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:02.797286 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:02.797256 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:03.169726 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:03.169698 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:37:03.194854 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:03.194824 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:03.196000 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:03.195982 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:10.182359 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:10.182309 2577 patch_prober.go:28] interesting pod/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 03 10:37:10.182783 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:10.182398 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 03 10:37:11.245758 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:11.244549 2577 generic.go:356] "Generic (PLEG): container finished" podID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerID="4bdfc47405bcd5e95726d8afade3f87bae9f4505624b7bdc87b5dd6bf2d21c37" exitCode=0 Jul 03 10:37:11.245758 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:11.244622 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" event={"ID":"a32ec802-99b7-44ab-aebe-c42b5572d653","Type":"ContainerDied","Data":"4bdfc47405bcd5e95726d8afade3f87bae9f4505624b7bdc87b5dd6bf2d21c37"} Jul 03 10:37:11.246408 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:11.246226 2577 scope.go:117] "RemoveContainer" containerID="4bdfc47405bcd5e95726d8afade3f87bae9f4505624b7bdc87b5dd6bf2d21c37" Jul 03 10:37:11.247604 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:11.247523 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:37:11.247684 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:11.247619 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:37:11.247745 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:11.247702 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:37:12.256076 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:12.256038 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" event={"ID":"a32ec802-99b7-44ab-aebe-c42b5572d653","Type":"ContainerStarted","Data":"568025db5c6f6f76dcd41b7c6684552845e70f925765b9f632d3892ad28b842f"} Jul 03 10:37:12.336075 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:12.336035 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:37:12.336422 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:12.336390 2577 patch_prober.go:28] interesting pod/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 03 10:37:12.336538 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:12.336453 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 03 10:37:13.336599 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:13.336548 2577 patch_prober.go:28] interesting pod/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 03 10:37:13.337161 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:13.336625 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 03 10:37:14.336456 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:14.336405 2577 patch_prober.go:28] interesting pod/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 03 10:37:14.336675 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:14.336480 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 03 10:37:15.336771 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:15.336723 2577 patch_prober.go:28] interesting pod/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 03 10:37:15.337298 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:15.336799 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 03 10:37:16.342596 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:16.342557 2577 patch_prober.go:28] interesting pod/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 03 10:37:16.343235 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:16.343204 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 03 10:37:17.340654 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:17.340619 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:37:17.341036 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:17.341020 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:37:17.341878 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:17.341859 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:37:18.196708 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:18.196669 2577 patch_prober.go:28] interesting pod/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 03 10:37:18.197180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:18.196723 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 03 10:37:18.293238 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:18.293206 2577 generic.go:356] "Generic (PLEG): container finished" podID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerID="8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1" exitCode=0 Jul 03 10:37:18.293451 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:18.293275 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" event={"ID":"c73fad2f-56d6-4bb6-8874-587bc39bc91d","Type":"ContainerDied","Data":"8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1"} Jul 03 10:37:18.293700 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:18.293678 2577 scope.go:117] "RemoveContainer" containerID="8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1" Jul 03 10:37:18.294751 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:18.294724 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:37:18.294808 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:18.294797 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:37:18.294844 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:18.294831 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:37:19.301250 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:19.301213 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" event={"ID":"c73fad2f-56d6-4bb6-8874-587bc39bc91d","Type":"ContainerStarted","Data":"a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0"} Jul 03 10:37:19.791183 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:19.791152 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:19.791367 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:19.791346 2577 patch_prober.go:28] interesting pod/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 03 10:37:19.791419 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:19.791392 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 03 10:37:20.791794 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:20.791757 2577 patch_prober.go:28] interesting pod/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 03 10:37:20.792242 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:20.791815 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 03 10:37:21.792165 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:21.792130 2577 patch_prober.go:28] interesting pod/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 03 10:37:21.792571 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:21.792185 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 03 10:37:22.791983 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:22.791945 2577 patch_prober.go:28] interesting pod/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 03 10:37:22.792219 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:22.792016 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 03 10:37:23.795183 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:23.795152 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:23.795636 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:23.795448 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:37:23.796009 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:37:23.795992 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:38:18.685774 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:18.685683 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc"] Jul 03 10:38:18.686314 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:18.686234 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" podUID="0de1a249-9a0c-467a-8efd-fc01b9b8d9bd" containerName="maas-api" containerID="cri-o://c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3" gracePeriod=30 Jul 03 10:38:18.939075 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:18.939009 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:38:18.972518 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:18.972492 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-maas-api-tls\") pod \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " Jul 03 10:38:18.972686 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:18.972542 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj9r5\" (UniqueName: \"kubernetes.io/projected/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-kube-api-access-fj9r5\") pod \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " Jul 03 10:38:18.972686 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:18.972609 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-openshift-service-ca\") pod \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\" (UID: \"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd\") " Jul 03 10:38:18.973017 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:18.972992 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0de1a249-9a0c-467a-8efd-fc01b9b8d9bd" (UID: "0de1a249-9a0c-467a-8efd-fc01b9b8d9bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:38:18.974627 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:18.974601 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-kube-api-access-fj9r5" (OuterVolumeSpecName: "kube-api-access-fj9r5") pod "0de1a249-9a0c-467a-8efd-fc01b9b8d9bd" (UID: "0de1a249-9a0c-467a-8efd-fc01b9b8d9bd"). InnerVolumeSpecName "kube-api-access-fj9r5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:38:18.974762 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:18.974625 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0de1a249-9a0c-467a-8efd-fc01b9b8d9bd" (UID: "0de1a249-9a0c-467a-8efd-fc01b9b8d9bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:38:19.073542 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.073503 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:38:19.073542 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.073533 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj9r5\" (UniqueName: \"kubernetes.io/projected/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-kube-api-access-fj9r5\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:38:19.073542 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.073542 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:38:19.614955 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.614916 2577 generic.go:356] "Generic (PLEG): container finished" podID="0de1a249-9a0c-467a-8efd-fc01b9b8d9bd" containerID="c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3" exitCode=0 Jul 03 10:38:19.615170 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.614961 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" Jul 03 10:38:19.615170 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.614999 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" event={"ID":"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd","Type":"ContainerDied","Data":"c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3"} Jul 03 10:38:19.615170 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.615050 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc" event={"ID":"0de1a249-9a0c-467a-8efd-fc01b9b8d9bd","Type":"ContainerDied","Data":"3f9d90ec79b7617d7bdb6c4f52d9817ea3dda11c23dfc179899eff8892985e12"} Jul 03 10:38:19.615170 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.615072 2577 scope.go:117] "RemoveContainer" containerID="c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3" Jul 03 10:38:19.625584 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.625563 2577 scope.go:117] "RemoveContainer" containerID="c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3" Jul 03 10:38:19.625890 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:38:19.625869 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3\": container with ID starting with c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3 not found: ID does not exist" containerID="c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3" Jul 03 10:38:19.625967 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.625898 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3"} err="failed to get container status \"c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3\": rpc error: code = NotFound desc = could not find container \"c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3\": container with ID starting with c45cada0bbfc7af43bc45681ffe145e7b8bbb5e39a604866c2ac60b1cc0801e3 not found: ID does not exist" Jul 03 10:38:19.641304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.641275 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc"] Jul 03 10:38:19.647455 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:19.647431 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d628f-b94f994f6-2glvc"] Jul 03 10:38:20.472195 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:38:20.472165 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0de1a249-9a0c-467a-8efd-fc01b9b8d9bd" path="/var/lib/kubelet/pods/0de1a249-9a0c-467a-8efd-fc01b9b8d9bd/volumes" Jul 03 10:39:01.733940 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:01.733903 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68"] Jul 03 10:39:01.734544 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:01.734135 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" containerID="cri-o://568025db5c6f6f76dcd41b7c6684552845e70f925765b9f632d3892ad28b842f" gracePeriod=30 Jul 03 10:39:02.343318 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:02.343278 2577 patch_prober.go:28] interesting pod/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 03 10:39:02.343499 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:02.343371 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 10:39:06.876492 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:06.876456 2577 generic.go:356] "Generic (PLEG): container finished" podID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerID="568025db5c6f6f76dcd41b7c6684552845e70f925765b9f632d3892ad28b842f" exitCode=0 Jul 03 10:39:06.876952 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:06.876524 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" event={"ID":"a32ec802-99b7-44ab-aebe-c42b5572d653","Type":"ContainerDied","Data":"568025db5c6f6f76dcd41b7c6684552845e70f925765b9f632d3892ad28b842f"} Jul 03 10:39:06.876952 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:06.876597 2577 scope.go:117] "RemoveContainer" containerID="4bdfc47405bcd5e95726d8afade3f87bae9f4505624b7bdc87b5dd6bf2d21c37" Jul 03 10:39:06.996910 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:06.996849 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:39:07.124113 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124059 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32ec802-99b7-44ab-aebe-c42b5572d653-istiod-ca-cert\") pod \"a32ec802-99b7-44ab-aebe-c42b5572d653\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " Jul 03 10:39:07.124305 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124162 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-token\") pod \"a32ec802-99b7-44ab-aebe-c42b5572d653\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " Jul 03 10:39:07.124305 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124184 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-podinfo\") pod \"a32ec802-99b7-44ab-aebe-c42b5572d653\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " Jul 03 10:39:07.124305 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124202 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-envoy\") pod \"a32ec802-99b7-44ab-aebe-c42b5572d653\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " Jul 03 10:39:07.124305 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124220 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-certs\") pod \"a32ec802-99b7-44ab-aebe-c42b5572d653\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " Jul 03 10:39:07.124305 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124292 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-socket\") pod \"a32ec802-99b7-44ab-aebe-c42b5572d653\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " Jul 03 10:39:07.124581 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124315 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-data\") pod \"a32ec802-99b7-44ab-aebe-c42b5572d653\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " Jul 03 10:39:07.124581 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124338 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4226l\" (UniqueName: \"kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-kube-api-access-4226l\") pod \"a32ec802-99b7-44ab-aebe-c42b5572d653\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " Jul 03 10:39:07.124581 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124376 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-credential-socket\") pod \"a32ec802-99b7-44ab-aebe-c42b5572d653\" (UID: \"a32ec802-99b7-44ab-aebe-c42b5572d653\") " Jul 03 10:39:07.124581 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124496 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a32ec802-99b7-44ab-aebe-c42b5572d653-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a32ec802-99b7-44ab-aebe-c42b5572d653" (UID: "a32ec802-99b7-44ab-aebe-c42b5572d653"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:39:07.124787 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124669 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a32ec802-99b7-44ab-aebe-c42b5572d653" (UID: "a32ec802-99b7-44ab-aebe-c42b5572d653"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:07.124860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124834 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:07.124860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124846 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a32ec802-99b7-44ab-aebe-c42b5572d653" (UID: "a32ec802-99b7-44ab-aebe-c42b5572d653"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:07.124979 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124859 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32ec802-99b7-44ab-aebe-c42b5572d653-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:07.124979 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.124902 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a32ec802-99b7-44ab-aebe-c42b5572d653" (UID: "a32ec802-99b7-44ab-aebe-c42b5572d653"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:07.125403 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.125369 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-data" (OuterVolumeSpecName: "istio-data") pod "a32ec802-99b7-44ab-aebe-c42b5572d653" (UID: "a32ec802-99b7-44ab-aebe-c42b5572d653"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:07.126629 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.126594 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-token" (OuterVolumeSpecName: "istio-token") pod "a32ec802-99b7-44ab-aebe-c42b5572d653" (UID: "a32ec802-99b7-44ab-aebe-c42b5572d653"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:07.126629 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.126602 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a32ec802-99b7-44ab-aebe-c42b5572d653" (UID: "a32ec802-99b7-44ab-aebe-c42b5572d653"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:39:07.126766 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.126648 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-kube-api-access-4226l" (OuterVolumeSpecName: "kube-api-access-4226l") pod "a32ec802-99b7-44ab-aebe-c42b5572d653" (UID: "a32ec802-99b7-44ab-aebe-c42b5572d653"). InnerVolumeSpecName "kube-api-access-4226l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:07.126766 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.126670 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a32ec802-99b7-44ab-aebe-c42b5572d653" (UID: "a32ec802-99b7-44ab-aebe-c42b5572d653"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:07.226032 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.225991 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:07.226032 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.226025 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:07.226032 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.226034 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:07.226032 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.226042 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:07.226342 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.226051 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:07.226342 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.226058 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4226l\" (UniqueName: \"kubernetes.io/projected/a32ec802-99b7-44ab-aebe-c42b5572d653-kube-api-access-4226l\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:07.226342 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.226067 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32ec802-99b7-44ab-aebe-c42b5572d653-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:07.883010 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.882979 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" Jul 03 10:39:07.883010 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.882998 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68" event={"ID":"a32ec802-99b7-44ab-aebe-c42b5572d653","Type":"ContainerDied","Data":"442c92bf3f7301503d2ad58330e6c485ff5a89400afd0b96241ecd8bd4c95bb6"} Jul 03 10:39:07.883508 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.883050 2577 scope.go:117] "RemoveContainer" containerID="568025db5c6f6f76dcd41b7c6684552845e70f925765b9f632d3892ad28b842f" Jul 03 10:39:07.909256 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.909212 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68"] Jul 03 10:39:07.911490 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:07.911465 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1d628f-openshift-default-848678f755-5jg68"] Jul 03 10:39:08.472174 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:08.472141 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" path="/var/lib/kubelet/pods/a32ec802-99b7-44ab-aebe-c42b5572d653/volumes" Jul 03 10:39:09.480688 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:09.480654 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7"] Jul 03 10:39:09.481141 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:09.480871 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" containerID="cri-o://a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0" gracePeriod=30 Jul 03 10:39:10.407664 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.407624 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7"] Jul 03 10:39:10.408173 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.408154 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" Jul 03 10:39:10.408227 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.408175 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" Jul 03 10:39:10.408227 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.408189 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0de1a249-9a0c-467a-8efd-fc01b9b8d9bd" containerName="maas-api" Jul 03 10:39:10.408227 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.408195 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0de1a249-9a0c-467a-8efd-fc01b9b8d9bd" containerName="maas-api" Jul 03 10:39:10.408325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.408271 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" Jul 03 10:39:10.408325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.408280 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0de1a249-9a0c-467a-8efd-fc01b9b8d9bd" containerName="maas-api" Jul 03 10:39:10.408325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.408287 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" Jul 03 10:39:10.412662 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.412646 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.418733 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.418712 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b8b776-openshift-default-dockercfg-xzl8b\"" Jul 03 10:39:10.432377 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.432342 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7"] Jul 03 10:39:10.457543 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.457500 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-podinfo\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.457728 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.457551 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-credential-socket\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.457728 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.457586 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-envoy\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.457728 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.457633 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-data\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.457728 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.457670 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5c5m\" (UniqueName: \"kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-kube-api-access-h5c5m\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.457884 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.457772 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-certs\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.457884 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.457801 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-token\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.457884 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.457868 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-socket\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.457981 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.457892 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aba6a79b-8bc5-4bea-8018-6ac0014965be-istiod-ca-cert\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.558951 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.558845 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-certs\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.558951 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.558884 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-token\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.558951 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.558925 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-socket\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.559595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.558954 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aba6a79b-8bc5-4bea-8018-6ac0014965be-istiod-ca-cert\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.559595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.559000 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-podinfo\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.559595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.559027 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-credential-socket\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.559595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.559058 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-envoy\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.559595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.559086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-data\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.559595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.559321 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5c5m\" (UniqueName: \"kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-kube-api-access-h5c5m\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.559595 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.559375 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-certs\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.560018 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.559675 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-data\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.560079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.560042 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aba6a79b-8bc5-4bea-8018-6ac0014965be-istiod-ca-cert\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.560866 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.560844 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-credential-socket\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.561042 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.560875 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-socket\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.562477 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.562427 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-envoy\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.565535 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.565479 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-podinfo\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.569593 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.569530 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-token\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.570233 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.570207 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5c5m\" (UniqueName: \"kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-kube-api-access-h5c5m\") pod \"e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.722767 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.722730 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:10.868989 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.868956 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7"] Jul 03 10:39:10.869987 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:39:10.869964 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba6a79b_8bc5_4bea_8018_6ac0014965be.slice/crio-e147e7764c4fab16f744f91d71fc3d639577a27a557b3548e0eff9252caf8299 WatchSource:0}: Error finding container e147e7764c4fab16f744f91d71fc3d639577a27a557b3548e0eff9252caf8299: Status 404 returned error can't find the container with id e147e7764c4fab16f744f91d71fc3d639577a27a557b3548e0eff9252caf8299 Jul 03 10:39:10.872154 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.872121 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:10.872218 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.872190 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:10.872253 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.872220 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:10.901025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:10.900996 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" event={"ID":"aba6a79b-8bc5-4bea-8018-6ac0014965be","Type":"ContainerStarted","Data":"e147e7764c4fab16f744f91d71fc3d639577a27a557b3548e0eff9252caf8299"} Jul 03 10:39:11.907849 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:11.907812 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" event={"ID":"aba6a79b-8bc5-4bea-8018-6ac0014965be","Type":"ContainerStarted","Data":"f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116"} Jul 03 10:39:11.947387 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:11.947321 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" podStartSLOduration=1.9473054730000001 podStartE2EDuration="1.947305473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:39:11.945166433 +0000 UTC m=+2030.011241622" watchObservedRunningTime="2026-07-03 10:39:11.947305473 +0000 UTC m=+2030.013380715" Jul 03 10:39:12.723734 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:12.723690 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:12.728792 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:12.728761 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:12.913579 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:12.913550 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:12.914451 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:12.914432 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:14.742409 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.742347 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:39:14.798984 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.798957 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-certs\") pod \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " Jul 03 10:39:14.799180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.799009 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istiod-ca-cert\") pod \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " Jul 03 10:39:14.799180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.799038 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-data\") pod \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " Jul 03 10:39:14.799180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.799106 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sxmt\" (UniqueName: \"kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-kube-api-access-2sxmt\") pod \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " Jul 03 10:39:14.799180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.799153 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-socket\") pod \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " Jul 03 10:39:14.799180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.799177 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-envoy\") pod \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " Jul 03 10:39:14.799463 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.799242 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-podinfo\") pod \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " Jul 03 10:39:14.799463 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.799275 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-token\") pod \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " Jul 03 10:39:14.799463 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.799301 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-credential-socket\") pod \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\" (UID: \"c73fad2f-56d6-4bb6-8874-587bc39bc91d\") " Jul 03 10:39:14.805063 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.799267 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c73fad2f-56d6-4bb6-8874-587bc39bc91d" (UID: "c73fad2f-56d6-4bb6-8874-587bc39bc91d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:14.805063 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.800266 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c73fad2f-56d6-4bb6-8874-587bc39bc91d" (UID: "c73fad2f-56d6-4bb6-8874-587bc39bc91d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:14.805063 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.800335 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c73fad2f-56d6-4bb6-8874-587bc39bc91d" (UID: "c73fad2f-56d6-4bb6-8874-587bc39bc91d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:39:14.805063 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.800382 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c73fad2f-56d6-4bb6-8874-587bc39bc91d" (UID: "c73fad2f-56d6-4bb6-8874-587bc39bc91d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:14.805063 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.803065 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-data" (OuterVolumeSpecName: "istio-data") pod "c73fad2f-56d6-4bb6-8874-587bc39bc91d" (UID: "c73fad2f-56d6-4bb6-8874-587bc39bc91d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:14.810045 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.808692 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c73fad2f-56d6-4bb6-8874-587bc39bc91d" (UID: "c73fad2f-56d6-4bb6-8874-587bc39bc91d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:39:14.810223 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.810174 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-token" (OuterVolumeSpecName: "istio-token") pod "c73fad2f-56d6-4bb6-8874-587bc39bc91d" (UID: "c73fad2f-56d6-4bb6-8874-587bc39bc91d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:14.810713 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.810569 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c73fad2f-56d6-4bb6-8874-587bc39bc91d" (UID: "c73fad2f-56d6-4bb6-8874-587bc39bc91d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:14.811517 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.811489 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-kube-api-access-2sxmt" (OuterVolumeSpecName: "kube-api-access-2sxmt") pod "c73fad2f-56d6-4bb6-8874-587bc39bc91d" (UID: "c73fad2f-56d6-4bb6-8874-587bc39bc91d"). InnerVolumeSpecName "kube-api-access-2sxmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:14.901272 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.901226 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:14.901272 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.901265 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:14.901272 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.901278 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sxmt\" (UniqueName: \"kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-kube-api-access-2sxmt\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:14.901547 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.901292 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:14.901547 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.901304 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:14.901547 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.901315 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:14.901547 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.901325 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73fad2f-56d6-4bb6-8874-587bc39bc91d-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:14.901547 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.901336 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:14.901547 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.901346 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73fad2f-56d6-4bb6-8874-587bc39bc91d-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:14.925837 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.925805 2577 generic.go:356] "Generic (PLEG): container finished" podID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerID="a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0" exitCode=0 Jul 03 10:39:14.926027 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.925876 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" Jul 03 10:39:14.926027 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.925886 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" event={"ID":"c73fad2f-56d6-4bb6-8874-587bc39bc91d","Type":"ContainerDied","Data":"a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0"} Jul 03 10:39:14.926027 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.925971 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7" event={"ID":"c73fad2f-56d6-4bb6-8874-587bc39bc91d","Type":"ContainerDied","Data":"6897ac1b2c3812e030f0e5c7a6ac7cd9b14bf65f0bb8b96ed46eb8774fafce32"} Jul 03 10:39:14.926027 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.925995 2577 scope.go:117] "RemoveContainer" containerID="a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0" Jul 03 10:39:14.935650 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.935626 2577 scope.go:117] "RemoveContainer" containerID="8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1" Jul 03 10:39:14.945060 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.945038 2577 scope.go:117] "RemoveContainer" containerID="a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0" Jul 03 10:39:14.945411 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:39:14.945386 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0\": container with ID starting with a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0 not found: ID does not exist" containerID="a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0" Jul 03 10:39:14.945521 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.945418 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0"} err="failed to get container status \"a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0\": rpc error: code = NotFound desc = could not find container \"a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0\": container with ID starting with a8a5cd53a97ab37ba3e2313af60cd4b436c89bfb9e0af21b6185da9e6ed0f5e0 not found: ID does not exist" Jul 03 10:39:14.945521 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.945446 2577 scope.go:117] "RemoveContainer" containerID="8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1" Jul 03 10:39:14.945709 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:39:14.945690 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1\": container with ID starting with 8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1 not found: ID does not exist" containerID="8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1" Jul 03 10:39:14.945761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.945713 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1"} err="failed to get container status \"8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1\": rpc error: code = NotFound desc = could not find container \"8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1\": container with ID starting with 8bad6847fb95c4a8272fa29a8f2e0a40beffa49977061d81858d1bff7e4149e1 not found: ID does not exist" Jul 03 10:39:14.957354 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.957324 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7"] Jul 03 10:39:14.959906 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:14.959881 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7436d1-openshift-default-6d7bf479b5-dt5x7"] Jul 03 10:39:16.471857 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:16.471826 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" path="/var/lib/kubelet/pods/c73fad2f-56d6-4bb6-8874-587bc39bc91d/volumes" Jul 03 10:39:19.298375 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.298313 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl"] Jul 03 10:39:19.299693 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.299664 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" Jul 03 10:39:19.299864 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.299852 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" Jul 03 10:39:19.299951 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.299941 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" Jul 03 10:39:19.300023 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.300014 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a32ec802-99b7-44ab-aebe-c42b5572d653" containerName="istio-proxy" Jul 03 10:39:19.300126 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.300114 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" Jul 03 10:39:19.300238 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.300227 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" Jul 03 10:39:19.300528 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.300513 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" Jul 03 10:39:19.300635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.300624 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c73fad2f-56d6-4bb6-8874-587bc39bc91d" containerName="istio-proxy" Jul 03 10:39:19.306304 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.306279 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.309996 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.309977 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-84ecdb-openshift-default-dockercfg-gpmhz\"" Jul 03 10:39:19.317146 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.317121 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl"] Jul 03 10:39:19.439849 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.439800 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-credential-socket\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.440028 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.439873 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-envoy\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.440028 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.439897 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-data\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.440028 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.439923 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-certs\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.440028 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.439949 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-istio-token\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.440202 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.440035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djpv8\" (UniqueName: \"kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-kube-api-access-djpv8\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.440202 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.440072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e159550c-f565-4d99-add1-45a0d50d1716-istiod-ca-cert\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.440202 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.440126 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e159550c-f565-4d99-add1-45a0d50d1716-istio-podinfo\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.440202 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.440141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-socket\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.541616 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.541571 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-certs\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.541616 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.541620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-istio-token\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.541897 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.541692 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djpv8\" (UniqueName: \"kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-kube-api-access-djpv8\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.541897 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.541722 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e159550c-f565-4d99-add1-45a0d50d1716-istiod-ca-cert\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.541897 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.541758 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e159550c-f565-4d99-add1-45a0d50d1716-istio-podinfo\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.541897 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.541781 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-socket\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.541897 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.541806 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-credential-socket\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.541897 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.541885 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-envoy\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.542220 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.541910 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-data\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.542220 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.542136 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-certs\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.542341 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.542304 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-credential-socket\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.542396 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.542356 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-data\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.542466 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.542446 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e159550c-f565-4d99-add1-45a0d50d1716-istiod-ca-cert\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.542578 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.542561 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-socket\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.544407 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.544386 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-envoy\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.544558 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.544540 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e159550c-f565-4d99-add1-45a0d50d1716-istio-podinfo\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.551217 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.551150 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-istio-token\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.551217 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.551162 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djpv8\" (UniqueName: \"kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-kube-api-access-djpv8\") pod \"e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.616922 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.616889 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:19.764126 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.764079 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl"] Jul 03 10:39:19.764872 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:39:19.764849 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode159550c_f565_4d99_add1_45a0d50d1716.slice/crio-59e7ee8e3caf72dc577f25bba139d16e18e49b1190c6646fdd95a1e2b3d74e38 WatchSource:0}: Error finding container 59e7ee8e3caf72dc577f25bba139d16e18e49b1190c6646fdd95a1e2b3d74e38: Status 404 returned error can't find the container with id 59e7ee8e3caf72dc577f25bba139d16e18e49b1190c6646fdd95a1e2b3d74e38 Jul 03 10:39:19.767015 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.766981 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:19.767113 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.767052 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:19.767113 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.767086 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:19.958606 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.958563 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" event={"ID":"e159550c-f565-4d99-add1-45a0d50d1716","Type":"ContainerStarted","Data":"e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272"} Jul 03 10:39:19.958813 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.958627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" event={"ID":"e159550c-f565-4d99-add1-45a0d50d1716","Type":"ContainerStarted","Data":"59e7ee8e3caf72dc577f25bba139d16e18e49b1190c6646fdd95a1e2b3d74e38"} Jul 03 10:39:19.987015 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:19.986945 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" podStartSLOduration=0.986928735 podStartE2EDuration="986.928735ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:39:19.985412181 +0000 UTC m=+2038.051487374" watchObservedRunningTime="2026-07-03 10:39:19.986928735 +0000 UTC m=+2038.053003927" Jul 03 10:39:20.617706 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:20.617660 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:20.622764 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:20.622738 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:20.964764 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:20.964736 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:20.965662 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:20.965637 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:27.914722 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:27.914689 2577 patch_prober.go:28] interesting pod/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 03 10:39:27.915151 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:27.914741 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 03 10:39:28.004358 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:28.004328 2577 generic.go:356] "Generic (PLEG): container finished" podID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerID="f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116" exitCode=0 Jul 03 10:39:28.004536 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:28.004397 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" event={"ID":"aba6a79b-8bc5-4bea-8018-6ac0014965be","Type":"ContainerDied","Data":"f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116"} Jul 03 10:39:28.004816 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:28.004801 2577 scope.go:117] "RemoveContainer" containerID="f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116" Jul 03 10:39:28.005856 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:28.005826 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:28.005931 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:28.005912 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:28.005967 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:28.005959 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:29.011317 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:29.011277 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" event={"ID":"aba6a79b-8bc5-4bea-8018-6ac0014965be","Type":"ContainerStarted","Data":"3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593"} Jul 03 10:39:29.723180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:29.723141 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:29.723376 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:29.723348 2577 patch_prober.go:28] interesting pod/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 03 10:39:29.723425 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:29.723394 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 03 10:39:30.724079 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:30.724037 2577 patch_prober.go:28] interesting pod/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 03 10:39:30.724611 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:30.724128 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 03 10:39:31.723692 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:31.723657 2577 patch_prober.go:28] interesting pod/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 03 10:39:31.723985 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:31.723722 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 03 10:39:32.723860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:32.723825 2577 patch_prober.go:28] interesting pod/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 03 10:39:32.724337 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:32.723892 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 03 10:39:33.727797 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:33.727763 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:33.728224 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:33.728110 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:33.728772 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:33.728755 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:37.006323 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:37.006275 2577 patch_prober.go:28] interesting pod/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 03 10:39:37.006751 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:37.006370 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 03 10:39:38.065648 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:38.065615 2577 generic.go:356] "Generic (PLEG): container finished" podID="e159550c-f565-4d99-add1-45a0d50d1716" containerID="e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272" exitCode=0 Jul 03 10:39:38.066151 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:38.065687 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" event={"ID":"e159550c-f565-4d99-add1-45a0d50d1716","Type":"ContainerDied","Data":"e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272"} Jul 03 10:39:38.066215 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:38.066186 2577 scope.go:117] "RemoveContainer" containerID="e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272" Jul 03 10:39:38.067350 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:38.067322 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:38.067426 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:38.067407 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:38.067471 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:38.067447 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:39.072818 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:39.072784 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" event={"ID":"e159550c-f565-4d99-add1-45a0d50d1716","Type":"ContainerStarted","Data":"d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc"} Jul 03 10:39:39.617637 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:39.617598 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:39.617889 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:39.617866 2577 patch_prober.go:28] interesting pod/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 03 10:39:39.617981 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:39.617924 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 03 10:39:40.618191 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:40.618161 2577 patch_prober.go:28] interesting pod/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 03 10:39:40.618666 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:40.618642 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 03 10:39:41.259961 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:41.258376 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7"] Jul 03 10:39:41.259961 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:41.258614 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" containerID="cri-o://3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593" gracePeriod=30 Jul 03 10:39:41.617875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:41.617792 2577 patch_prober.go:28] interesting pod/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 03 10:39:41.617875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:41.617846 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 03 10:39:42.618287 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:42.618249 2577 patch_prober.go:28] interesting pod/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 03 10:39:42.618716 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:42.618306 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 03 10:39:43.618039 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:43.618002 2577 patch_prober.go:28] interesting pod/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 03 10:39:43.618247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:43.618060 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 03 10:39:44.620809 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:44.620725 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:44.621319 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:44.621120 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:44.621749 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:44.621728 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:46.520431 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.520397 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:46.601062 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.600972 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5c5m\" (UniqueName: \"kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-kube-api-access-h5c5m\") pod \"aba6a79b-8bc5-4bea-8018-6ac0014965be\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " Jul 03 10:39:46.601062 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601012 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-token\") pod \"aba6a79b-8bc5-4bea-8018-6ac0014965be\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " Jul 03 10:39:46.601062 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601048 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-credential-socket\") pod \"aba6a79b-8bc5-4bea-8018-6ac0014965be\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " Jul 03 10:39:46.601360 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601073 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-data\") pod \"aba6a79b-8bc5-4bea-8018-6ac0014965be\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " Jul 03 10:39:46.601360 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601115 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-certs\") pod \"aba6a79b-8bc5-4bea-8018-6ac0014965be\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " Jul 03 10:39:46.601360 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601137 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aba6a79b-8bc5-4bea-8018-6ac0014965be-istiod-ca-cert\") pod \"aba6a79b-8bc5-4bea-8018-6ac0014965be\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " Jul 03 10:39:46.601360 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601186 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-envoy\") pod \"aba6a79b-8bc5-4bea-8018-6ac0014965be\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " Jul 03 10:39:46.601360 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601232 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-podinfo\") pod \"aba6a79b-8bc5-4bea-8018-6ac0014965be\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " Jul 03 10:39:46.601360 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601259 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-socket\") pod \"aba6a79b-8bc5-4bea-8018-6ac0014965be\" (UID: \"aba6a79b-8bc5-4bea-8018-6ac0014965be\") " Jul 03 10:39:46.601683 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601420 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aba6a79b-8bc5-4bea-8018-6ac0014965be" (UID: "aba6a79b-8bc5-4bea-8018-6ac0014965be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:46.601683 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601446 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aba6a79b-8bc5-4bea-8018-6ac0014965be" (UID: "aba6a79b-8bc5-4bea-8018-6ac0014965be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:46.601683 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601574 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:46.601683 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601594 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:46.601683 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601645 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aba6a79b-8bc5-4bea-8018-6ac0014965be" (UID: "aba6a79b-8bc5-4bea-8018-6ac0014965be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:46.601909 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601683 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aba6a79b-8bc5-4bea-8018-6ac0014965be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aba6a79b-8bc5-4bea-8018-6ac0014965be" (UID: "aba6a79b-8bc5-4bea-8018-6ac0014965be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:39:46.601991 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.601973 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-data" (OuterVolumeSpecName: "istio-data") pod "aba6a79b-8bc5-4bea-8018-6ac0014965be" (UID: "aba6a79b-8bc5-4bea-8018-6ac0014965be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:46.603788 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.603763 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-token" (OuterVolumeSpecName: "istio-token") pod "aba6a79b-8bc5-4bea-8018-6ac0014965be" (UID: "aba6a79b-8bc5-4bea-8018-6ac0014965be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:46.603908 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.603769 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-kube-api-access-h5c5m" (OuterVolumeSpecName: "kube-api-access-h5c5m") pod "aba6a79b-8bc5-4bea-8018-6ac0014965be" (UID: "aba6a79b-8bc5-4bea-8018-6ac0014965be"). InnerVolumeSpecName "kube-api-access-h5c5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:46.603968 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.603935 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aba6a79b-8bc5-4bea-8018-6ac0014965be" (UID: "aba6a79b-8bc5-4bea-8018-6ac0014965be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:46.604036 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.604019 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aba6a79b-8bc5-4bea-8018-6ac0014965be" (UID: "aba6a79b-8bc5-4bea-8018-6ac0014965be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:39:46.701995 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.701956 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:46.701995 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.701987 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aba6a79b-8bc5-4bea-8018-6ac0014965be-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:46.701995 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.701995 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:46.702273 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.702022 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:46.702273 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.702033 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba6a79b-8bc5-4bea-8018-6ac0014965be-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:46.702273 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.702046 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5c5m\" (UniqueName: \"kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-kube-api-access-h5c5m\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:46.702273 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:46.702061 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aba6a79b-8bc5-4bea-8018-6ac0014965be-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:47.120988 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.120954 2577 generic.go:356] "Generic (PLEG): container finished" podID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerID="3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593" exitCode=0 Jul 03 10:39:47.121187 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.120998 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" Jul 03 10:39:47.121187 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.121042 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" event={"ID":"aba6a79b-8bc5-4bea-8018-6ac0014965be","Type":"ContainerDied","Data":"3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593"} Jul 03 10:39:47.121187 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.121120 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7" event={"ID":"aba6a79b-8bc5-4bea-8018-6ac0014965be","Type":"ContainerDied","Data":"e147e7764c4fab16f744f91d71fc3d639577a27a557b3548e0eff9252caf8299"} Jul 03 10:39:47.121187 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.121143 2577 scope.go:117] "RemoveContainer" containerID="3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593" Jul 03 10:39:47.130696 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.130674 2577 scope.go:117] "RemoveContainer" containerID="f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116" Jul 03 10:39:47.138896 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.138874 2577 scope.go:117] "RemoveContainer" containerID="3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593" Jul 03 10:39:47.139178 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:39:47.139159 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593\": container with ID starting with 3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593 not found: ID does not exist" containerID="3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593" Jul 03 10:39:47.139234 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.139187 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593"} err="failed to get container status \"3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593\": rpc error: code = NotFound desc = could not find container \"3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593\": container with ID starting with 3fccf4e75db4a1884a50f1239658c48eaa5ad0891cc6b26ab51f109c8cb58593 not found: ID does not exist" Jul 03 10:39:47.139234 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.139209 2577 scope.go:117] "RemoveContainer" containerID="f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116" Jul 03 10:39:47.139419 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:39:47.139401 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116\": container with ID starting with f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116 not found: ID does not exist" containerID="f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116" Jul 03 10:39:47.139462 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.139425 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116"} err="failed to get container status \"f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116\": rpc error: code = NotFound desc = could not find container \"f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116\": container with ID starting with f352b51ddc92a101af154aa59555fba7ef65f4a3a71903418accba0420e9c116 not found: ID does not exist" Jul 03 10:39:47.145678 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.145654 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7"] Jul 03 10:39:47.151597 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:47.151574 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b8b776-openshift-default-64cd5846b7-c6xd7"] Jul 03 10:39:48.472139 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:48.472091 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" path="/var/lib/kubelet/pods/aba6a79b-8bc5-4bea-8018-6ac0014965be/volumes" Jul 03 10:39:48.523080 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:48.523040 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl"] Jul 03 10:39:48.523344 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:48.523319 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" containerID="cri-o://d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc" gracePeriod=30 Jul 03 10:39:49.372642 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.372603 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2"] Jul 03 10:39:49.373141 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.373117 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" Jul 03 10:39:49.373141 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.373138 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" Jul 03 10:39:49.373348 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.373151 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" Jul 03 10:39:49.373348 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.373157 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" Jul 03 10:39:49.373348 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.373217 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" Jul 03 10:39:49.376152 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.376130 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.379249 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.379194 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-cda536-openshift-default-dockercfg-92cb7\"" Jul 03 10:39:49.389089 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.389065 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2"] Jul 03 10:39:49.428565 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.428510 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-token\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.428767 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.428640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-credential-socket\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.428767 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.428686 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv7f4\" (UniqueName: \"kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-kube-api-access-bv7f4\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.428767 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.428715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-data\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.428767 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.428740 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-socket\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.428956 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.428783 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-certs\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.428956 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.428816 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-podinfo\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.428956 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.428899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-envoy\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.428956 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.428936 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istiod-ca-cert\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.529390 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.529358 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-podinfo\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.529858 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.529419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-envoy\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.529858 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.529457 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istiod-ca-cert\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.529858 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.529534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-token\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.529858 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.529590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-credential-socket\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.529858 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.529742 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv7f4\" (UniqueName: \"kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-kube-api-access-bv7f4\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.529858 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.529829 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-data\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.529858 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.529861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-socket\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.530352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.529888 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-certs\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.530352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.529974 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-credential-socket\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.530352 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.530205 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-data\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.530518 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.530407 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-socket\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.530518 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.530423 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-certs\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.530822 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.530799 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istiod-ca-cert\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.531892 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.531868 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-envoy\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.532035 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.532017 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-podinfo\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.538738 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.538707 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-token\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.538986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.538952 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv7f4\" (UniqueName: \"kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-kube-api-access-bv7f4\") pod \"e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.686168 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.686115 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:49.832325 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.832292 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2"] Jul 03 10:39:49.833842 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:39:49.833804 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ede34aa_1e10_4f4b_bfe3_faa20a7eef86.slice/crio-fe95842540b8a8da6aff831f8fd4a6da416b01c486be1daf4c0405bf5f4f9cdd WatchSource:0}: Error finding container fe95842540b8a8da6aff831f8fd4a6da416b01c486be1daf4c0405bf5f4f9cdd: Status 404 returned error can't find the container with id fe95842540b8a8da6aff831f8fd4a6da416b01c486be1daf4c0405bf5f4f9cdd Jul 03 10:39:49.835930 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.835891 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:49.836000 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.835980 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:49.836052 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:49.836039 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:50.140599 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:50.140505 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" event={"ID":"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86","Type":"ContainerStarted","Data":"2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5"} Jul 03 10:39:50.140599 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:50.140565 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" event={"ID":"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86","Type":"ContainerStarted","Data":"fe95842540b8a8da6aff831f8fd4a6da416b01c486be1daf4c0405bf5f4f9cdd"} Jul 03 10:39:50.164889 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:50.164835 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" podStartSLOduration=1.16482006 podStartE2EDuration="1.16482006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:39:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:39:50.161515514 +0000 UTC m=+2068.227590699" watchObservedRunningTime="2026-07-03 10:39:50.16482006 +0000 UTC m=+2068.230895251" Jul 03 10:39:50.687080 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:50.687048 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:50.692620 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:50.692593 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:51.145862 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:51.145824 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:51.146906 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:51.146890 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:39:53.792228 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.792162 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:53.870033 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.869996 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djpv8\" (UniqueName: \"kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-kube-api-access-djpv8\") pod \"e159550c-f565-4d99-add1-45a0d50d1716\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " Jul 03 10:39:53.870033 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870035 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-socket\") pod \"e159550c-f565-4d99-add1-45a0d50d1716\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " Jul 03 10:39:53.870318 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870087 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-data\") pod \"e159550c-f565-4d99-add1-45a0d50d1716\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " Jul 03 10:39:53.870318 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870144 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-certs\") pod \"e159550c-f565-4d99-add1-45a0d50d1716\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " Jul 03 10:39:53.870318 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870209 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-envoy\") pod \"e159550c-f565-4d99-add1-45a0d50d1716\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " Jul 03 10:39:53.870318 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870247 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-istio-token\") pod \"e159550c-f565-4d99-add1-45a0d50d1716\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " Jul 03 10:39:53.870539 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870333 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e159550c-f565-4d99-add1-45a0d50d1716-istio-podinfo\") pod \"e159550c-f565-4d99-add1-45a0d50d1716\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " Jul 03 10:39:53.870539 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870380 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-credential-socket\") pod \"e159550c-f565-4d99-add1-45a0d50d1716\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " Jul 03 10:39:53.870539 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870364 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e159550c-f565-4d99-add1-45a0d50d1716" (UID: "e159550c-f565-4d99-add1-45a0d50d1716"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:53.870539 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870419 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e159550c-f565-4d99-add1-45a0d50d1716-istiod-ca-cert\") pod \"e159550c-f565-4d99-add1-45a0d50d1716\" (UID: \"e159550c-f565-4d99-add1-45a0d50d1716\") " Jul 03 10:39:53.870734 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870675 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:53.870787 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.870758 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e159550c-f565-4d99-add1-45a0d50d1716" (UID: "e159550c-f565-4d99-add1-45a0d50d1716"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:53.871076 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.871054 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e159550c-f565-4d99-add1-45a0d50d1716" (UID: "e159550c-f565-4d99-add1-45a0d50d1716"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:53.871190 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.871078 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e159550c-f565-4d99-add1-45a0d50d1716-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e159550c-f565-4d99-add1-45a0d50d1716" (UID: "e159550c-f565-4d99-add1-45a0d50d1716"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:39:53.871190 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.871120 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-data" (OuterVolumeSpecName: "istio-data") pod "e159550c-f565-4d99-add1-45a0d50d1716" (UID: "e159550c-f565-4d99-add1-45a0d50d1716"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:53.872503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.872474 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e159550c-f565-4d99-add1-45a0d50d1716" (UID: "e159550c-f565-4d99-add1-45a0d50d1716"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:39:53.872503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.872482 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-istio-token" (OuterVolumeSpecName: "istio-token") pod "e159550c-f565-4d99-add1-45a0d50d1716" (UID: "e159550c-f565-4d99-add1-45a0d50d1716"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:53.872779 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.872755 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e159550c-f565-4d99-add1-45a0d50d1716-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e159550c-f565-4d99-add1-45a0d50d1716" (UID: "e159550c-f565-4d99-add1-45a0d50d1716"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:39:53.872851 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.872763 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-kube-api-access-djpv8" (OuterVolumeSpecName: "kube-api-access-djpv8") pod "e159550c-f565-4d99-add1-45a0d50d1716" (UID: "e159550c-f565-4d99-add1-45a0d50d1716"). InnerVolumeSpecName "kube-api-access-djpv8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:53.971841 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.971795 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:53.971841 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.971836 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:53.971841 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.971848 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e159550c-f565-4d99-add1-45a0d50d1716-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:53.972128 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.971860 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:53.972128 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.971872 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e159550c-f565-4d99-add1-45a0d50d1716-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:53.972128 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.971885 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djpv8\" (UniqueName: \"kubernetes.io/projected/e159550c-f565-4d99-add1-45a0d50d1716-kube-api-access-djpv8\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:53.972128 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.971895 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:53.972128 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:53.971908 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e159550c-f565-4d99-add1-45a0d50d1716-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:39:54.162782 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.162751 2577 generic.go:356] "Generic (PLEG): container finished" podID="e159550c-f565-4d99-add1-45a0d50d1716" containerID="d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc" exitCode=0 Jul 03 10:39:54.162998 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.162807 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" Jul 03 10:39:54.162998 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.162841 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" event={"ID":"e159550c-f565-4d99-add1-45a0d50d1716","Type":"ContainerDied","Data":"d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc"} Jul 03 10:39:54.162998 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.162895 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl" event={"ID":"e159550c-f565-4d99-add1-45a0d50d1716","Type":"ContainerDied","Data":"59e7ee8e3caf72dc577f25bba139d16e18e49b1190c6646fdd95a1e2b3d74e38"} Jul 03 10:39:54.162998 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.162919 2577 scope.go:117] "RemoveContainer" containerID="d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc" Jul 03 10:39:54.175395 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.174715 2577 scope.go:117] "RemoveContainer" containerID="e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272" Jul 03 10:39:54.183982 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.183959 2577 scope.go:117] "RemoveContainer" containerID="d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc" Jul 03 10:39:54.184313 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:39:54.184296 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc\": container with ID starting with d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc not found: ID does not exist" containerID="d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc" Jul 03 10:39:54.184372 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.184321 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc"} err="failed to get container status \"d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc\": rpc error: code = NotFound desc = could not find container \"d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc\": container with ID starting with d6286f2792e3372f05e94282f2c1340d9896411945593d4e3e745292143abadc not found: ID does not exist" Jul 03 10:39:54.184372 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.184337 2577 scope.go:117] "RemoveContainer" containerID="e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272" Jul 03 10:39:54.184634 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:39:54.184608 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272\": container with ID starting with e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272 not found: ID does not exist" containerID="e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272" Jul 03 10:39:54.184725 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.184643 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272"} err="failed to get container status \"e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272\": rpc error: code = NotFound desc = could not find container \"e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272\": container with ID starting with e34a035c69cfa116f37a9f51c44bb1801d6ab57bc9998c1e8bb8bf8637202272 not found: ID does not exist" Jul 03 10:39:54.187986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.187963 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl"] Jul 03 10:39:54.191956 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.191928 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-84ecdb-openshift-default-56bb5b7b6f-cljtl"] Jul 03 10:39:54.471954 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:54.471871 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e159550c-f565-4d99-add1-45a0d50d1716" path="/var/lib/kubelet/pods/e159550c-f565-4d99-add1-45a0d50d1716/volumes" Jul 03 10:39:57.260262 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.260231 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv"] Jul 03 10:39:57.260670 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.260590 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" Jul 03 10:39:57.260670 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.260601 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" Jul 03 10:39:57.260670 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.260670 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aba6a79b-8bc5-4bea-8018-6ac0014965be" containerName="istio-proxy" Jul 03 10:39:57.260781 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.260679 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" Jul 03 10:39:57.260781 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.260685 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" Jul 03 10:39:57.262954 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.262933 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.266335 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.266310 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-62c45e-openshift-default-dockercfg-kj6f2\"" Jul 03 10:39:57.280877 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.280851 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv"] Jul 03 10:39:57.304193 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.304159 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-data\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.304383 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.304221 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr5g7\" (UniqueName: \"kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-kube-api-access-qr5g7\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.304383 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.304269 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a19c0bce-4479-4df0-b537-6ab8c7a13960-istiod-ca-cert\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.304383 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.304304 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-socket\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.304383 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.304352 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-credential-socket\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.304533 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.304387 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-token\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.304533 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.304450 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-certs\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.304533 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.304477 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-podinfo\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.304630 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.304526 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-envoy\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.405750 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.405714 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-envoy\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.405911 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.405756 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-data\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.405911 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.405813 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr5g7\" (UniqueName: \"kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-kube-api-access-qr5g7\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.405911 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.405846 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a19c0bce-4479-4df0-b537-6ab8c7a13960-istiod-ca-cert\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.405911 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.405882 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-socket\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.406180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.405923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-credential-socket\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.406180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.405959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-token\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.406180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.406029 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-certs\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.406180 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.406071 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-podinfo\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.406382 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.406322 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-data\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.406679 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.406602 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-credential-socket\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.406679 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.406646 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-socket\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.406679 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.406670 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-certs\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.406679 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.406675 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a19c0bce-4479-4df0-b537-6ab8c7a13960-istiod-ca-cert\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.408216 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.408195 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-envoy\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.408574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.408553 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-podinfo\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.417298 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.417276 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr5g7\" (UniqueName: \"kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-kube-api-access-qr5g7\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.417419 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.417295 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-token\") pod \"e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.573405 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.573304 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:57.744225 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.742657 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:57.744225 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.742754 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:57.744225 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.742798 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:39:57.762986 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:57.762942 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv"] Jul 03 10:39:58.187677 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:58.187636 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" event={"ID":"a19c0bce-4479-4df0-b537-6ab8c7a13960","Type":"ContainerStarted","Data":"7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed"} Jul 03 10:39:58.187839 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:58.187701 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" event={"ID":"a19c0bce-4479-4df0-b537-6ab8c7a13960","Type":"ContainerStarted","Data":"3ea09232016a3a2abf199448b4c44a539ad6915afb8bd127a48fe3157a23c081"} Jul 03 10:39:58.209555 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:58.209500 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" podStartSLOduration=1.209482972 podStartE2EDuration="1.209482972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:39:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:39:58.209185563 +0000 UTC m=+2076.275260755" watchObservedRunningTime="2026-07-03 10:39:58.209482972 +0000 UTC m=+2076.275558166" Jul 03 10:39:58.574334 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:58.574235 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:58.579741 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:58.579704 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:59.192623 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:59.192587 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:39:59.193708 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:39:59.193690 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:40:05.229422 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:05.229390 2577 generic.go:356] "Generic (PLEG): container finished" podID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerID="2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5" exitCode=0 Jul 03 10:40:05.229855 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:05.229466 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" event={"ID":"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86","Type":"ContainerDied","Data":"2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5"} Jul 03 10:40:05.229899 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:05.229857 2577 scope.go:117] "RemoveContainer" containerID="2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5" Jul 03 10:40:05.230910 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:05.230883 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:40:05.230981 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:05.230947 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:40:05.230981 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:05.230977 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:40:06.146189 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:06.146142 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:40:06.241001 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:06.240954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" event={"ID":"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86","Type":"ContainerStarted","Data":"2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a"} Jul 03 10:40:06.687163 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:06.687119 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:40:06.688444 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:06.688413 2577 patch_prober.go:28] interesting pod/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 03 10:40:06.688568 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:06.688467 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 03 10:40:07.687208 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:07.687169 2577 patch_prober.go:28] interesting pod/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 03 10:40:07.687601 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:07.687241 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 03 10:40:08.687380 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:08.687349 2577 patch_prober.go:28] interesting pod/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 03 10:40:08.687931 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:08.687903 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 03 10:40:09.687432 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:09.687392 2577 patch_prober.go:28] interesting pod/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 03 10:40:09.687934 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:09.687462 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 03 10:40:10.691060 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:10.691027 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:40:10.691497 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:10.691362 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:40:10.692072 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:10.692054 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:40:13.297275 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:13.297242 2577 generic.go:356] "Generic (PLEG): container finished" podID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerID="7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed" exitCode=0 Jul 03 10:40:13.297702 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:13.297321 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" event={"ID":"a19c0bce-4479-4df0-b537-6ab8c7a13960","Type":"ContainerDied","Data":"7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed"} Jul 03 10:40:13.297748 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:13.297717 2577 scope.go:117] "RemoveContainer" containerID="7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed" Jul 03 10:40:13.298780 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:13.298752 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:40:13.298853 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:13.298825 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:40:13.298891 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:13.298861 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 10:40:14.192541 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:14.192494 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:40:14.304487 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:14.304455 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" event={"ID":"a19c0bce-4479-4df0-b537-6ab8c7a13960","Type":"ContainerStarted","Data":"fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71"} Jul 03 10:40:14.574584 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:14.574480 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:40:14.576321 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:14.575924 2577 patch_prober.go:28] interesting pod/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 03 10:40:14.576321 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:14.576003 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 03 10:40:15.574390 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:15.574350 2577 patch_prober.go:28] interesting pod/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 03 10:40:15.574789 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:15.574411 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 03 10:40:16.573907 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:16.573872 2577 patch_prober.go:28] interesting pod/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 03 10:40:16.574121 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:16.573925 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 03 10:40:17.574455 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:17.574416 2577 patch_prober.go:28] interesting pod/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 03 10:40:17.574853 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:17.574482 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 03 10:40:18.574404 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:18.574364 2577 patch_prober.go:28] interesting pod/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 03 10:40:18.574606 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:18.574427 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 03 10:40:19.577720 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:19.577685 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:40:19.578247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:19.578059 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:40:19.578839 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:19.578818 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:40:22.547186 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:22.547156 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:40:22.551902 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:22.551873 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:40:22.555032 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:22.555007 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:40:22.559511 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:40:22.559492 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:45:22.582943 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:45:22.582910 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:45:22.587612 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:45:22.587588 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:45:22.591966 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:45:22.591949 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:45:22.595792 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:45:22.595776 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hb8xm_25536018-295a-44e5-b396-44d6f533d2b2/ovn-acl-logging/0.log" Jul 03 10:46:40.464808 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:40.464765 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2"] Jul 03 10:46:40.465369 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:40.465009 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerName="istio-proxy" containerID="cri-o://2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a" gracePeriod=30 Jul 03 10:46:40.693773 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:40.693731 2577 patch_prober.go:28] interesting pod/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 03 10:46:40.693937 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:40.693800 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 10:46:45.723027 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.722996 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:46:45.909677 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.909649 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-certs\") pod \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " Jul 03 10:46:45.909870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.909704 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv7f4\" (UniqueName: \"kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-kube-api-access-bv7f4\") pod \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " Jul 03 10:46:45.909870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.909730 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-data\") pod \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " Jul 03 10:46:45.909870 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.909753 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-envoy\") pod \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " Jul 03 10:46:45.910036 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.909866 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istiod-ca-cert\") pod \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " Jul 03 10:46:45.910036 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.909915 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-podinfo\") pod \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " Jul 03 10:46:45.910036 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.910008 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-token\") pod \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " Jul 03 10:46:45.910036 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.910009 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" (UID: "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:45.910284 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.910123 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-credential-socket\") pod \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " Jul 03 10:46:45.910284 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.910184 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-socket\") pod \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\" (UID: \"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86\") " Jul 03 10:46:45.910384 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.910281 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" (UID: "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:46:45.910523 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.910504 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:45.910523 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.910513 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" (UID: "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:45.910636 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.910530 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:45.910636 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.910580 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" (UID: "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:45.910706 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.910668 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" (UID: "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:45.912162 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.912132 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-kube-api-access-bv7f4" (OuterVolumeSpecName: "kube-api-access-bv7f4") pod "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" (UID: "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86"). InnerVolumeSpecName "kube-api-access-bv7f4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:46:45.912162 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.912126 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" (UID: "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:46:45.912435 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.912417 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" (UID: "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:45.912564 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:45.912545 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" (UID: "8ede34aa-1e10-4f4b-bfe3-faa20a7eef86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:46:46.010993 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.010954 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:46.010993 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.010985 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:46.010993 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.010994 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:46.011270 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.011005 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv7f4\" (UniqueName: \"kubernetes.io/projected/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-kube-api-access-bv7f4\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:46.011270 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.011015 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:46.011270 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.011023 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:46.011270 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.011030 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:46.403702 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.403668 2577 generic.go:356] "Generic (PLEG): container finished" podID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerID="2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a" exitCode=0 Jul 03 10:46:46.403905 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.403712 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" Jul 03 10:46:46.403905 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.403719 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" event={"ID":"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86","Type":"ContainerDied","Data":"2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a"} Jul 03 10:46:46.403905 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.403771 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2" event={"ID":"8ede34aa-1e10-4f4b-bfe3-faa20a7eef86","Type":"ContainerDied","Data":"fe95842540b8a8da6aff831f8fd4a6da416b01c486be1daf4c0405bf5f4f9cdd"} Jul 03 10:46:46.403905 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.403800 2577 scope.go:117] "RemoveContainer" containerID="2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a" Jul 03 10:46:46.414635 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.414613 2577 scope.go:117] "RemoveContainer" containerID="2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5" Jul 03 10:46:46.423436 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.423415 2577 scope.go:117] "RemoveContainer" containerID="2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a" Jul 03 10:46:46.423699 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:46:46.423679 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a\": container with ID starting with 2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a not found: ID does not exist" containerID="2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a" Jul 03 10:46:46.423743 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.423708 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a"} err="failed to get container status \"2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a\": rpc error: code = NotFound desc = could not find container \"2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a\": container with ID starting with 2200cf8705fef5bd58857ced056cc41b3b5299c1b0a9240af1102d0849569b9a not found: ID does not exist" Jul 03 10:46:46.423743 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.423724 2577 scope.go:117] "RemoveContainer" containerID="2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5" Jul 03 10:46:46.423926 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:46:46.423906 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5\": container with ID starting with 2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5 not found: ID does not exist" containerID="2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5" Jul 03 10:46:46.423962 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.423934 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5"} err="failed to get container status \"2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5\": rpc error: code = NotFound desc = could not find container \"2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5\": container with ID starting with 2e34cc78881c118c6f66526f23c9638956d4aacbdcc6a8564ea209bf055673d5 not found: ID does not exist" Jul 03 10:46:46.429038 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.429013 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2"] Jul 03 10:46:46.431905 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.431882 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-cda536-openshift-default-7f5bbdf66c-7f6z2"] Jul 03 10:46:46.472202 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:46.472173 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" path="/var/lib/kubelet/pods/8ede34aa-1e10-4f4b-bfe3-faa20a7eef86/volumes" Jul 03 10:46:47.905448 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:47.905419 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv"] Jul 03 10:46:47.905891 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:47.905646 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" containerID="cri-o://fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71" gracePeriod=30 Jul 03 10:46:49.581058 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:49.581011 2577 patch_prober.go:28] interesting pod/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 03 10:46:49.581489 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:49.581080 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 10:46:53.164944 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.164917 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:46:53.174656 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174575 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-socket\") pod \"a19c0bce-4479-4df0-b537-6ab8c7a13960\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " Jul 03 10:46:53.174656 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174624 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-certs\") pod \"a19c0bce-4479-4df0-b537-6ab8c7a13960\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " Jul 03 10:46:53.174860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174671 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-token\") pod \"a19c0bce-4479-4df0-b537-6ab8c7a13960\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " Jul 03 10:46:53.174860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174701 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-podinfo\") pod \"a19c0bce-4479-4df0-b537-6ab8c7a13960\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " Jul 03 10:46:53.174860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174740 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr5g7\" (UniqueName: \"kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-kube-api-access-qr5g7\") pod \"a19c0bce-4479-4df0-b537-6ab8c7a13960\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " Jul 03 10:46:53.174860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174784 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-data\") pod \"a19c0bce-4479-4df0-b537-6ab8c7a13960\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " Jul 03 10:46:53.174860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174819 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a19c0bce-4479-4df0-b537-6ab8c7a13960-istiod-ca-cert\") pod \"a19c0bce-4479-4df0-b537-6ab8c7a13960\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " Jul 03 10:46:53.174860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174858 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-credential-socket\") pod \"a19c0bce-4479-4df0-b537-6ab8c7a13960\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " Jul 03 10:46:53.175195 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174884 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a19c0bce-4479-4df0-b537-6ab8c7a13960" (UID: "a19c0bce-4479-4df0-b537-6ab8c7a13960"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:53.175195 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174898 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-envoy\") pod \"a19c0bce-4479-4df0-b537-6ab8c7a13960\" (UID: \"a19c0bce-4479-4df0-b537-6ab8c7a13960\") " Jul 03 10:46:53.175195 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.174991 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a19c0bce-4479-4df0-b537-6ab8c7a13960" (UID: "a19c0bce-4479-4df0-b537-6ab8c7a13960"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:53.175356 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.175261 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a19c0bce-4479-4df0-b537-6ab8c7a13960-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a19c0bce-4479-4df0-b537-6ab8c7a13960" (UID: "a19c0bce-4479-4df0-b537-6ab8c7a13960"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:46:53.175356 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.175273 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:53.175356 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.175295 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-workload-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:53.175526 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.175506 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a19c0bce-4479-4df0-b537-6ab8c7a13960" (UID: "a19c0bce-4479-4df0-b537-6ab8c7a13960"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:53.176003 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.175974 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-data" (OuterVolumeSpecName: "istio-data") pod "a19c0bce-4479-4df0-b537-6ab8c7a13960" (UID: "a19c0bce-4479-4df0-b537-6ab8c7a13960"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:53.176993 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.176962 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a19c0bce-4479-4df0-b537-6ab8c7a13960" (UID: "a19c0bce-4479-4df0-b537-6ab8c7a13960"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:53.177126 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.177037 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-kube-api-access-qr5g7" (OuterVolumeSpecName: "kube-api-access-qr5g7") pod "a19c0bce-4479-4df0-b537-6ab8c7a13960" (UID: "a19c0bce-4479-4df0-b537-6ab8c7a13960"). InnerVolumeSpecName "kube-api-access-qr5g7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:46:53.177126 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.177071 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a19c0bce-4479-4df0-b537-6ab8c7a13960" (UID: "a19c0bce-4479-4df0-b537-6ab8c7a13960"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 10:46:53.177356 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.177339 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-token" (OuterVolumeSpecName: "istio-token") pod "a19c0bce-4479-4df0-b537-6ab8c7a13960" (UID: "a19c0bce-4479-4df0-b537-6ab8c7a13960"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:46:53.276003 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.275950 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:53.276003 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.275998 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-podinfo\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:53.276003 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.276007 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr5g7\" (UniqueName: \"kubernetes.io/projected/a19c0bce-4479-4df0-b537-6ab8c7a13960-kube-api-access-qr5g7\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:53.276003 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.276016 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-data\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:53.276003 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.276025 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a19c0bce-4479-4df0-b537-6ab8c7a13960-istiod-ca-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:53.276361 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.276033 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-credential-socket\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:53.276361 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.276042 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a19c0bce-4479-4df0-b537-6ab8c7a13960-istio-envoy\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\"" Jul 03 10:46:53.333199 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.333164 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5cfb4c9d6c-dsjg4_9af36f4b-77ef-493a-aa78-a27337e3a40d/maas-api/0.log" Jul 03 10:46:53.442203 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.442087 2577 generic.go:356] "Generic (PLEG): container finished" podID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerID="fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71" exitCode=0 Jul 03 10:46:53.442203 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.442147 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" Jul 03 10:46:53.442388 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.442151 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" event={"ID":"a19c0bce-4479-4df0-b537-6ab8c7a13960","Type":"ContainerDied","Data":"fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71"} Jul 03 10:46:53.442388 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.442261 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv" event={"ID":"a19c0bce-4479-4df0-b537-6ab8c7a13960","Type":"ContainerDied","Data":"3ea09232016a3a2abf199448b4c44a539ad6915afb8bd127a48fe3157a23c081"} Jul 03 10:46:53.442388 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.442287 2577 scope.go:117] "RemoveContainer" containerID="fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71" Jul 03 10:46:53.460792 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.460769 2577 scope.go:117] "RemoveContainer" containerID="7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed" Jul 03 10:46:53.469736 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.469708 2577 scope.go:117] "RemoveContainer" containerID="fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71" Jul 03 10:46:53.470073 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:46:53.470049 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71\": container with ID starting with fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71 not found: ID does not exist" containerID="fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71" Jul 03 10:46:53.470208 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.470085 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71"} err="failed to get container status \"fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71\": rpc error: code = NotFound desc = could not find container \"fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71\": container with ID starting with fdc14236619517654106f54cf84257c486ce66abdf23550e7bf5f62ed631be71 not found: ID does not exist" Jul 03 10:46:53.470208 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.470129 2577 scope.go:117] "RemoveContainer" containerID="7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed" Jul 03 10:46:53.470781 ip-10-0-135-154 kubenswrapper[2577]: E0703 10:46:53.470750 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed\": container with ID starting with 7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed not found: ID does not exist" containerID="7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed" Jul 03 10:46:53.470866 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.470789 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed"} err="failed to get container status \"7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed\": rpc error: code = NotFound desc = could not find container \"7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed\": container with ID starting with 7f073096cd5d0a205877b807d389b47873781282e60b5c5905959916b1c8a0ed not found: ID does not exist" Jul 03 10:46:53.473127 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.473109 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv"] Jul 03 10:46:53.477510 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.477486 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-62c45e-openshift-default-74f658f9d7-5q8hv"] Jul 03 10:46:53.695538 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:53.695426 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b7966cdb9-hjdqs_c2ab595c-3236-4b10-853f-46f3f019bb48/manager/0.log" Jul 03 10:46:54.286625 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:54.286592 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-789547c8f-gn6c9_57c6e0fb-235a-4e66-8027-e3b3c282eb39/manager/0.log" Jul 03 10:46:54.293284 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:54.293258 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-789547c8f-gn6c9_57c6e0fb-235a-4e66-8027-e3b3c282eb39/copy-manifests/0.log" Jul 03 10:46:54.472574 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:54.472541 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" path="/var/lib/kubelet/pods/a19c0bce-4479-4df0-b537-6ab8c7a13960/volumes" Jul 03 10:46:55.185256 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.185223 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc_454a3546-cfbb-490a-a4b7-fa27d71d1f78/util/0.log" Jul 03 10:46:55.204656 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.204625 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc_454a3546-cfbb-490a-a4b7-fa27d71d1f78/pull/0.log" Jul 03 10:46:55.219217 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.219183 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc_454a3546-cfbb-490a-a4b7-fa27d71d1f78/extract/0.log" Jul 03 10:46:55.333829 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.333798 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh_d0d7020c-8bff-46fe-94f8-841f94df58ff/pull/0.log" Jul 03 10:46:55.344677 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.344639 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh_d0d7020c-8bff-46fe-94f8-841f94df58ff/extract/0.log" Jul 03 10:46:55.351391 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.351367 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh_d0d7020c-8bff-46fe-94f8-841f94df58ff/util/0.log" Jul 03 10:46:55.461989 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.461897 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc_32fc9888-034a-4c33-8eaf-08d777547812/util/0.log" Jul 03 10:46:55.469507 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.469474 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc_32fc9888-034a-4c33-8eaf-08d777547812/pull/0.log" Jul 03 10:46:55.476590 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.476568 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc_32fc9888-034a-4c33-8eaf-08d777547812/extract/0.log" Jul 03 10:46:55.591965 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.591935 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc_f6453d61-fdc7-45f3-b0fb-c8f145718e38/pull/0.log" Jul 03 10:46:55.605232 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.605206 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc_f6453d61-fdc7-45f3-b0fb-c8f145718e38/extract/0.log" Jul 03 10:46:55.611947 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:55.611922 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc_f6453d61-fdc7-45f3-b0fb-c8f145718e38/util/0.log" Jul 03 10:46:56.896235 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:56.896205 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bdbf6_5d3c99ce-9387-408f-abf4-0ac29400b5b7/istio-proxy/0.log" Jul 03 10:46:57.022327 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:57.022294 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5xtgx_d8a88800-bef0-4891-9a13-94caaa5d5ec8/discovery/0.log" Jul 03 10:46:57.376709 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:57.376602 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8pq9n_d509345f-1dc1-4756-ac38-8b41b968bee6/istio-proxy/1.log" Jul 03 10:46:57.739487 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:57.739457 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f7ff5477c-6gn75_7a741adb-6089-40c5-ba07-7f334eadd790/router/0.log" Jul 03 10:46:58.182685 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:58.182630 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-bss7q_f3a54bb6-65c8-4fc1-82f8-b19558618cae/main/0.log" Jul 03 10:46:58.408719 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:46:58.408673 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh78nm_0b78239f-e50a-49bf-afa7-8cc1ca170def/main/0.log" Jul 03 10:47:05.392641 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:05.392605 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vz6gj_108d4f92-ca93-47d8-a45f-ba7088301bfa/global-pull-secret-syncer/0.log" Jul 03 10:47:05.484566 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:05.484531 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rrn2q_ef4101c3-8dd9-450e-96d1-e7e8159cf062/konnectivity-agent/0.log" Jul 03 10:47:05.574265 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:05.574233 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-154.ec2.internal_40608af7edd160b369e43165ab923d25/haproxy/0.log" Jul 03 10:47:09.370047 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.370004 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc_454a3546-cfbb-490a-a4b7-fa27d71d1f78/extract/0.log" Jul 03 10:47:09.437684 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.437647 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc_454a3546-cfbb-490a-a4b7-fa27d71d1f78/util/0.log" Jul 03 10:47:09.482319 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.482289 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfjfc_454a3546-cfbb-490a-a4b7-fa27d71d1f78/pull/0.log" Jul 03 10:47:09.519111 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.519060 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh_d0d7020c-8bff-46fe-94f8-841f94df58ff/extract/0.log" Jul 03 10:47:09.553860 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.553831 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh_d0d7020c-8bff-46fe-94f8-841f94df58ff/util/0.log" Jul 03 10:47:09.593373 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.593322 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q54dh_d0d7020c-8bff-46fe-94f8-841f94df58ff/pull/0.log" Jul 03 10:47:09.640302 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.640210 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc_32fc9888-034a-4c33-8eaf-08d777547812/extract/0.log" Jul 03 10:47:09.679834 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.679806 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc_32fc9888-034a-4c33-8eaf-08d777547812/util/0.log" Jul 03 10:47:09.722453 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.722425 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738zxvc_32fc9888-034a-4c33-8eaf-08d777547812/pull/0.log" Jul 03 10:47:09.759025 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.758997 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc_f6453d61-fdc7-45f3-b0fb-c8f145718e38/extract/0.log" Jul 03 10:47:09.802810 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.802777 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc_f6453d61-fdc7-45f3-b0fb-c8f145718e38/util/0.log" Jul 03 10:47:09.843222 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:09.843188 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vnqqc_f6453d61-fdc7-45f3-b0fb-c8f145718e38/pull/0.log" Jul 03 10:47:12.835404 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:12.835310 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-c4dj5_6eac509e-8324-4a69-ae4d-7923496d16a0/cluster-monitoring-operator/0.log" Jul 03 10:47:13.096515 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:13.096410 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbvn4_eeb800ca-84ba-45c1-802a-c251a4cc7a1d/node-exporter/0.log" Jul 03 10:47:13.120118 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:13.120063 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbvn4_eeb800ca-84ba-45c1-802a-c251a4cc7a1d/kube-rbac-proxy/0.log" Jul 03 10:47:13.145483 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:13.145444 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbvn4_eeb800ca-84ba-45c1-802a-c251a4cc7a1d/init-textfile/0.log" Jul 03 10:47:13.537015 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:13.536982 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-lqcgw_4405e64a-4de1-4c24-b73e-a24b3e1f29e7/prometheus-operator/0.log" Jul 03 10:47:13.557073 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:13.557040 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-lqcgw_4405e64a-4de1-4c24-b73e-a24b3e1f29e7/kube-rbac-proxy/0.log" Jul 03 10:47:13.582945 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:13.582912 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-dsc85_f5df1fbc-077d-4cd4-a01a-c081349e9c25/prometheus-operator-admission-webhook/0.log" Jul 03 10:47:15.374411 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374329 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c"] Jul 03 10:47:15.374830 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374761 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" Jul 03 10:47:15.374830 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374777 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e159550c-f565-4d99-add1-45a0d50d1716" containerName="istio-proxy" Jul 03 10:47:15.374830 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374787 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" Jul 03 10:47:15.374830 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374793 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" Jul 03 10:47:15.374830 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374799 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerName="istio-proxy" Jul 03 10:47:15.374830 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374804 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerName="istio-proxy" Jul 03 10:47:15.374830 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374809 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" Jul 03 10:47:15.374830 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374815 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" Jul 03 10:47:15.375085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374886 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" Jul 03 10:47:15.375085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374894 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a19c0bce-4479-4df0-b537-6ab8c7a13960" containerName="istio-proxy" Jul 03 10:47:15.375085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374900 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerName="istio-proxy" Jul 03 10:47:15.375085 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.374907 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ede34aa-1e10-4f4b-bfe3-faa20a7eef86" containerName="istio-proxy" Jul 03 10:47:15.378033 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.378012 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.380837 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.380817 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dwvz5\"/\"kube-root-ca.crt\"" Jul 03 10:47:15.380956 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.380854 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dwvz5\"/\"default-dockercfg-zrqnd\"" Jul 03 10:47:15.382264 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.382246 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dwvz5\"/\"openshift-service-ca.crt\"" Jul 03 10:47:15.387669 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.387647 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c"] Jul 03 10:47:15.466503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.466465 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-lib-modules\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.466503 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.466506 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-sys\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.466729 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.466525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-proc\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.466729 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.466545 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkwjv\" (UniqueName: \"kubernetes.io/projected/96174f49-f38c-4895-8949-7b628d1670c2-kube-api-access-vkwjv\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.466729 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.466656 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-podres\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.567460 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.567429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-sys\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.567460 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.567459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-proc\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.567725 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.567480 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkwjv\" (UniqueName: \"kubernetes.io/projected/96174f49-f38c-4895-8949-7b628d1670c2-kube-api-access-vkwjv\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.567725 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.567542 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-podres\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.567725 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.567568 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-sys\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.567725 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.567582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-lib-modules\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.567725 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.567568 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-proc\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.567725 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.567698 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-lib-modules\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.567725 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.567696 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96174f49-f38c-4895-8949-7b628d1670c2-podres\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.575666 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.575641 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkwjv\" (UniqueName: \"kubernetes.io/projected/96174f49-f38c-4895-8949-7b628d1670c2-kube-api-access-vkwjv\") pod \"perf-node-gather-daemonset-8685c\" (UID: \"96174f49-f38c-4895-8949-7b628d1670c2\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.609780 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.609746 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/1.log" Jul 03 10:47:15.618360 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.618326 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8mlv5_8cc555a9-6421-474e-bcb7-399b574dc972/console-operator/2.log" Jul 03 10:47:15.688749 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.688715 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:15.828589 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:15.828556 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c"] Jul 03 10:47:15.830607 ip-10-0-135-154 kubenswrapper[2577]: W0703 10:47:15.830582 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod96174f49_f38c_4895_8949_7b628d1670c2.slice/crio-e3acfbb2dc11217e5d774b6e550cd58a5c5138a7b7f1f770c0a43c62de7308e8 WatchSource:0}: Error finding container e3acfbb2dc11217e5d774b6e550cd58a5c5138a7b7f1f770c0a43c62de7308e8: Status 404 returned error can't find the container with id e3acfbb2dc11217e5d774b6e550cd58a5c5138a7b7f1f770c0a43c62de7308e8 Jul 03 10:47:16.240253 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:16.240165 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-spvxp_a54488f8-1418-4d53-ace8-1120c1c8fe30/download-server/0.log" Jul 03 10:47:16.570247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:16.570148 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" event={"ID":"96174f49-f38c-4895-8949-7b628d1670c2","Type":"ContainerStarted","Data":"21d890655a8fdf67fd1c2ea60e5265a7f4bb31cc4d5b21178f57309652ee2462"} Jul 03 10:47:16.570247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:16.570203 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" event={"ID":"96174f49-f38c-4895-8949-7b628d1670c2","Type":"ContainerStarted","Data":"e3acfbb2dc11217e5d774b6e550cd58a5c5138a7b7f1f770c0a43c62de7308e8"} Jul 03 10:47:16.570247 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:16.570224 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:16.589034 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:16.588983 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" podStartSLOduration=1.588969938 podStartE2EDuration="1.588969938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:47:16.586880888 +0000 UTC m=+2514.652956086" watchObservedRunningTime="2026-07-03 10:47:16.588969938 +0000 UTC m=+2514.655045178" Jul 03 10:47:16.836968 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:16.836877 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-lqvkr_35d12807-d620-406c-ae20-d8a5ee68e92d/volume-data-source-validator/0.log" Jul 03 10:47:17.947283 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:17.947255 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bdd9n_efd9c9dc-f958-4414-bd48-3d2e6051d75c/dns/0.log" Jul 03 10:47:17.967761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:17.967734 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bdd9n_efd9c9dc-f958-4414-bd48-3d2e6051d75c/kube-rbac-proxy/0.log" Jul 03 10:47:18.088929 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:18.088898 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kzm94_75626273-af77-4838-99cc-2d3b274069d4/dns-node-resolver/0.log" Jul 03 10:47:18.634782 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:18.634751 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bm5md_01500b78-d50b-467c-bf74-f4b3e310e159/node-ca/0.log" Jul 03 10:47:19.595217 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:19.595190 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bdbf6_5d3c99ce-9387-408f-abf4-0ac29400b5b7/istio-proxy/0.log" Jul 03 10:47:19.784266 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:19.784233 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5xtgx_d8a88800-bef0-4891-9a13-94caaa5d5ec8/discovery/0.log" Jul 03 10:47:19.924988 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:19.924956 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8pq9n_d509345f-1dc1-4756-ac38-8b41b968bee6/istio-proxy/0.log" Jul 03 10:47:19.976541 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:19.976507 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8pq9n_d509345f-1dc1-4756-ac38-8b41b968bee6/istio-proxy/1.log" Jul 03 10:47:20.113851 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:20.113822 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f7ff5477c-6gn75_7a741adb-6089-40c5-ba07-7f334eadd790/router/0.log" Jul 03 10:47:20.786639 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:20.786609 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7xc64_3c24a47b-93f2-4783-9f16-e2fbf02d7ceb/serve-healthcheck-canary/0.log" Jul 03 10:47:21.414252 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:21.414220 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-tbrbs_0293a9da-1f5a-4a5a-8d5d-8619a7735b2f/insights-operator/0.log" Jul 03 10:47:21.414509 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:21.414488 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-tbrbs_0293a9da-1f5a-4a5a-8d5d-8619a7735b2f/insights-operator/1.log" Jul 03 10:47:21.567507 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:21.567478 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kqgvb_651ced07-9e27-481a-81a1-6df08a338b9f/kube-rbac-proxy/0.log" Jul 03 10:47:21.589132 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:21.589089 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kqgvb_651ced07-9e27-481a-81a1-6df08a338b9f/exporter/0.log" Jul 03 10:47:21.610366 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:21.610336 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kqgvb_651ced07-9e27-481a-81a1-6df08a338b9f/extractor/0.log" Jul 03 10:47:22.585739 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:22.585714 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-8685c" Jul 03 10:47:23.921736 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:23.921695 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5cfb4c9d6c-dsjg4_9af36f4b-77ef-493a-aa78-a27337e3a40d/maas-api/0.log" Jul 03 10:47:24.124857 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:24.124827 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b7966cdb9-hjdqs_c2ab595c-3236-4b10-853f-46f3f019bb48/manager/0.log" Jul 03 10:47:24.375345 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:24.375318 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-789547c8f-gn6c9_57c6e0fb-235a-4e66-8027-e3b3c282eb39/manager/0.log" Jul 03 10:47:24.395207 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:24.395179 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-789547c8f-gn6c9_57c6e0fb-235a-4e66-8027-e3b3c282eb39/copy-manifests/0.log" Jul 03 10:47:25.922295 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:25.922262 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bdbf6_5d3c99ce-9387-408f-abf4-0ac29400b5b7/istio-proxy/0.log" Jul 03 10:47:25.923283 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:25.923263 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5779fb96bf-rnrd4_bad9b19d-1e40-4fa3-a66b-e8e7f151a57d/manager/0.log" Jul 03 10:47:26.110182 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:26.110150 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5xtgx_d8a88800-bef0-4891-9a13-94caaa5d5ec8/discovery/0.log" Jul 03 10:47:26.155875 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:26.155836 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8pq9n_d509345f-1dc1-4756-ac38-8b41b968bee6/istio-proxy/0.log" Jul 03 10:47:26.209080 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:26.208984 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8pq9n_d509345f-1dc1-4756-ac38-8b41b968bee6/istio-proxy/1.log" Jul 03 10:47:26.303707 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:26.303631 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f7ff5477c-6gn75_7a741adb-6089-40c5-ba07-7f334eadd790/router/0.log" Jul 03 10:47:27.487577 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:27.487479 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bdbf6_5d3c99ce-9387-408f-abf4-0ac29400b5b7/istio-proxy/0.log" Jul 03 10:47:27.675392 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:27.675346 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5xtgx_d8a88800-bef0-4891-9a13-94caaa5d5ec8/discovery/0.log" Jul 03 10:47:27.725266 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:27.725237 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8pq9n_d509345f-1dc1-4756-ac38-8b41b968bee6/istio-proxy/0.log" Jul 03 10:47:27.779691 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:27.779606 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8pq9n_d509345f-1dc1-4756-ac38-8b41b968bee6/istio-proxy/1.log" Jul 03 10:47:27.862661 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:27.862630 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f7ff5477c-6gn75_7a741adb-6089-40c5-ba07-7f334eadd790/router/0.log" Jul 03 10:47:31.164060 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:31.164023 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-rhcp6_a7e396d0-1598-4e1b-8203-660c98a64ea7/kube-storage-version-migrator-operator/1.log" Jul 03 10:47:31.165761 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:31.165734 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-rhcp6_a7e396d0-1598-4e1b-8203-660c98a64ea7/kube-storage-version-migrator-operator/0.log" Jul 03 10:47:32.520142 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:32.520110 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gcpsk_3a89dafb-6d42-40f3-b9d4-cb4a0c839af6/kube-multus-additional-cni-plugins/0.log" Jul 03 10:47:32.541599 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:32.541563 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gcpsk_3a89dafb-6d42-40f3-b9d4-cb4a0c839af6/egress-router-binary-copy/0.log" Jul 03 10:47:32.563936 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:32.563909 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gcpsk_3a89dafb-6d42-40f3-b9d4-cb4a0c839af6/cni-plugins/0.log" Jul 03 10:47:32.586363 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:32.586327 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gcpsk_3a89dafb-6d42-40f3-b9d4-cb4a0c839af6/bond-cni-plugin/0.log" Jul 03 10:47:32.607989 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:32.607962 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gcpsk_3a89dafb-6d42-40f3-b9d4-cb4a0c839af6/routeoverride-cni/0.log" Jul 03 10:47:32.628993 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:32.628966 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gcpsk_3a89dafb-6d42-40f3-b9d4-cb4a0c839af6/whereabouts-cni-bincopy/0.log" Jul 03 10:47:32.649496 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:32.649470 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gcpsk_3a89dafb-6d42-40f3-b9d4-cb4a0c839af6/whereabouts-cni/0.log" Jul 03 10:47:32.717029 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:32.716998 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-g55zz_57e21cd6-3595-4064-95a5-2b22c0824be6/kube-multus/0.log" Jul 03 10:47:32.772075 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:32.771979 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2rjc5_ad84ff02-cefd-4c8e-b897-8a97a3727b79/network-metrics-daemon/0.log" Jul 03 10:47:32.790923 ip-10-0-135-154 kubenswrapper[2577]: I0703 10:47:32.790898 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2rjc5_ad84ff02-cefd-4c8e-b897-8a97a3727b79/kube-rbac-proxy/0.log"