Jul 08 09:06:40.457153 ip-10-0-136-68 systemd[1]: Starting Kubernetes Kubelet... Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.929805 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935518 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935538 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935544 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935548 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935555 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935559 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935562 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:06:40.956541 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935566 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935569 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935572 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935575 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935577 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935580 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935586 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935589 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935592 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935595 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935598 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935600 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935609 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935612 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935615 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935618 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935622 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935629 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935631 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935635 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:06:40.958828 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935637 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935640 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935647 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935650 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935653 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935655 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935658 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935661 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935667 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935670 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935673 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935678 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935682 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935688 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935691 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935695 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935698 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935701 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935708 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935711 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:06:40.960846 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935714 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935717 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935719 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935722 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935729 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935732 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935736 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935740 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935744 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935751 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935755 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935759 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935762 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935788 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935795 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935798 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935801 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935804 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935806 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:06:40.967435 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935809 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935812 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935814 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935821 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935823 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935826 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935829 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935832 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935834 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935841 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935845 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935847 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935850 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935853 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935860 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935864 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935866 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935871 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935874 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:06:40.970073 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935881 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935884 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935888 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935891 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935894 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935904 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935909 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935913 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935920 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935923 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935926 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935929 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935932 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.935935 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936855 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936867 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936871 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936875 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936878 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936881 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:06:40.972344 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936884 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936887 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936890 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936893 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936895 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936898 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936900 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936903 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936906 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936909 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936911 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936914 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936916 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936920 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936924 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936927 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936931 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936934 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936938 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936941 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:06:40.989760 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936943 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936946 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936948 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936951 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936954 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936956 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936959 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936961 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936964 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936967 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936969 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936972 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936974 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936977 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936980 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936982 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936985 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936987 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936990 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936992 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:06:40.993342 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936995 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.936998 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937001 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937004 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937007 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937011 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937015 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937018 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937020 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937023 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937026 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937029 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937031 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937034 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937036 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937039 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937042 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937048 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937050 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937053 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:06:41.006551 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937056 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937058 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937061 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937063 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937066 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937069 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937072 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937075 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937078 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937080 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937083 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937085 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937088 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937092 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937096 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937099 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937103 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937107 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937110 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:06:41.010057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937113 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937116 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937119 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937122 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937124 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937127 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937130 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937133 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937135 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937138 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937142 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937145 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937147 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937150 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937527 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937531 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937534 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937536 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937539 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:06:41.024129 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937542 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:06:41.016234 ip-10-0-136-68 systemd[1]: Started Kubernetes Kubelet. Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937545 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937548 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937550 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937553 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937556 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937558 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937561 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937564 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937567 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937569 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937572 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937574 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937577 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937579 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937582 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937585 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937588 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937590 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:06:41.026432 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937594 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937597 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937599 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937602 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937606 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937609 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937611 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937615 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937617 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937620 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937624 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937627 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937630 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937633 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937636 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937639 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937641 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937644 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937647 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937649 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:06:41.032450 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937652 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937654 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937657 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937659 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937662 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937664 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937667 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937670 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937673 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937675 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937678 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937681 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937683 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937686 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937688 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937691 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937694 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937697 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937701 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937704 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:06:41.037222 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937707 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937709 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937712 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937714 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937717 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937719 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937722 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937724 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937727 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937730 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937732 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937735 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937737 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937740 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937742 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937745 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937747 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937749 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937752 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937755 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:06:41.040030 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937758 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937760 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937763 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937780 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937782 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937785 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937788 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937790 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937794 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937797 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937801 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937804 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937807 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937810 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.937813 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937879 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937886 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937892 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937896 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937900 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937904 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 09:06:41.044374 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937909 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937913 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937916 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937920 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937923 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937926 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937931 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937934 2572 flags.go:64] FLAG: --cgroup-root="" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937937 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937940 2572 flags.go:64] FLAG: --client-ca-file="" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937944 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937947 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937950 2572 flags.go:64] FLAG: --cluster-domain="" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937953 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937956 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937964 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937968 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937972 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937975 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937980 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937983 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937986 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937989 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937992 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 09:06:41.046669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937995 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.937998 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938003 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938006 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938009 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938012 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938014 2572 flags.go:64] FLAG: --enable-server="true" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938018 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938022 2572 flags.go:64] FLAG: --event-burst="100" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938026 2572 flags.go:64] FLAG: --event-qps="50" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938029 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938032 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938035 2572 flags.go:64] FLAG: --eviction-hard="" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938039 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938046 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938049 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938052 2572 flags.go:64] FLAG: --eviction-soft="" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938055 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938058 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938061 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938064 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938067 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938070 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938073 2572 flags.go:64] FLAG: --feature-gates="" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938077 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 09:06:41.051754 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938080 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938083 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938089 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938092 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938095 2572 flags.go:64] FLAG: --help="false" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938097 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938101 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938104 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938106 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938110 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938113 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938116 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938119 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938122 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938125 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938128 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938131 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938134 2572 flags.go:64] FLAG: --kube-reserved="" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938137 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938139 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938142 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938146 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938149 2572 flags.go:64] FLAG: --lock-file="" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938152 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 09:06:41.207756 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938155 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938158 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938164 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938167 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938170 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938172 2572 flags.go:64] FLAG: --logging-format="text" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938175 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938178 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938182 2572 flags.go:64] FLAG: --manifest-url="" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938184 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938192 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938195 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938199 2572 flags.go:64] FLAG: --max-pods="110" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938202 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938206 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938209 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938212 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938215 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938218 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938221 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938228 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938231 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938234 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 09:06:41.226875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938238 2572 flags.go:64] FLAG: --pod-cidr="" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938241 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938246 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938249 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938252 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938255 2572 flags.go:64] FLAG: --port="10250" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938260 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938263 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0312b801127774e6a" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938267 2572 flags.go:64] FLAG: --qos-reserved="" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938270 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938273 2572 flags.go:64] FLAG: --register-node="true" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938276 2572 flags.go:64] FLAG: --register-with-taints="" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938279 2572 flags.go:64] FLAG: --registry-burst="10" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938282 2572 flags.go:64] FLAG: --registry-qps="5" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938285 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938288 2572 flags.go:64] FLAG: --reserved-memory="" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938292 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938295 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938299 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938301 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938306 2572 flags.go:64] FLAG: --runonce="false" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938309 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938312 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938315 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938318 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 09:06:41.229347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938320 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938323 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938327 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938330 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938332 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938335 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938338 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938341 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938344 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938347 2572 flags.go:64] FLAG: --system-cgroups="" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938350 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938355 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938358 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938363 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938366 2572 flags.go:64] FLAG: --tls-min-version="" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938369 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938372 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938375 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938378 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938381 2572 flags.go:64] FLAG: --v="2" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938385 2572 flags.go:64] FLAG: --version="false" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938389 2572 flags.go:64] FLAG: --vmodule="" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938393 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.938397 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938509 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:06:41.230346 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938512 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938517 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938522 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938525 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938528 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938531 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938534 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938536 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938539 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938542 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938544 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938547 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938549 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938552 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938555 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938558 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938560 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938563 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938566 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:06:41.233637 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938570 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938574 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938577 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938580 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938584 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938588 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938591 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938594 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938597 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938599 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938602 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938604 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938607 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938609 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938612 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938616 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938619 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938621 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938624 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:06:41.234507 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938626 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938629 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938632 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938634 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938637 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938639 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938641 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938644 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938647 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938649 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938652 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938654 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938657 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938659 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938663 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938665 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938668 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938671 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938673 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938676 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:06:41.236421 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938678 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938681 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938683 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938686 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938688 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938691 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938694 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938696 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938700 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938703 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938705 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938708 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938710 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938713 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938716 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938718 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938721 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938723 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938726 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938728 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:06:41.237049 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938731 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938733 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938736 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938738 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938741 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938743 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938747 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938750 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938753 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938755 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938758 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938760 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938763 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938781 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938784 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938786 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938789 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938791 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938794 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:06:41.238303 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.938797 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.939683 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.951047 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.951065 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.952246 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.952254 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952314 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952319 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952323 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952326 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952329 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952332 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952335 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952338 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:06:41.238920 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952341 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952345 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952349 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952352 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952355 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952358 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952362 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952365 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952367 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952371 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952374 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952377 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952380 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952384 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952388 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952395 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952398 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952401 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952404 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:06:41.239546 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952406 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952409 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952411 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952414 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952416 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952419 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952422 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952424 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952427 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952430 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952433 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952435 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952439 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952442 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952446 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952449 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952452 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952455 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952458 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952461 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:06:41.240158 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952464 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952467 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952469 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952472 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952474 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952477 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952480 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952482 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952486 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952489 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952491 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952494 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952496 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952499 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952502 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952504 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952507 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952510 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952512 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952515 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:06:41.240794 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952517 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952520 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952522 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952525 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952527 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952529 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952532 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952536 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952539 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952541 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952544 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952546 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952550 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952553 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952556 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952558 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952561 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952564 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952566 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952569 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:06:41.241686 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952574 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952576 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952580 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952582 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952585 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952588 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952590 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952593 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952596 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952598 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952601 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952603 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.952608 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952724 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952728 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:06:41.242389 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952731 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952735 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952739 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952743 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952746 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952750 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952752 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952755 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952758 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952760 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952778 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952783 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952786 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952789 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952791 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952794 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952797 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952801 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952804 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952806 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:06:41.242872 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952809 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952812 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952814 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952817 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952820 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952823 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952825 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952828 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952831 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952835 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952838 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952841 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952844 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952848 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952851 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952854 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952857 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952862 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952865 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:06:41.243518 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952868 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952871 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952873 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952876 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952880 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952882 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952885 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952888 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952890 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952893 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952897 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952900 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952902 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952905 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952907 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952910 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952913 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952915 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952918 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952920 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:06:41.244145 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952923 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952925 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952928 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952931 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952933 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952936 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952938 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952941 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952943 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952946 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952950 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952952 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952955 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952958 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952961 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952963 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952966 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952969 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952972 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952974 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:06:41.244981 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952977 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952980 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952985 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952988 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952991 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952994 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.952997 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953000 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953002 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953005 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953008 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953010 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953013 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953016 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953018 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953021 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953023 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:06:41.245610 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:40.953026 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.953031 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.953153 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.956419 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.958007 2572 server.go:1019] "Starting client certificate rotation" Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.958883 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.958927 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.961482 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.964422 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.973917 2572 log.go:25] "Validated CRI v1 runtime API" Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.981511 2572 log.go:25] "Validated CRI v1 image API" Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.982359 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 09:06:41.246272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.986274 2572 fs.go:135] Filesystem UUIDs: map[092965f2-7c5f-48d9-b5ef-e42f08aea983:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a6cd6e2a-dae6-41a5-9b4d-bdb7acf746c8:/dev/nvme0n1p3] Jul 08 09:06:41.246694 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.986293 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 09:06:41.246694 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.991281 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.992159 2572 manager.go:216] Machine: {Timestamp:2026-07-08 09:06:40.990574522 +0000 UTC m=+0.402516453 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100284 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28ad0b9e658663ecd03041ea6a19ba SystemUUID:ec28ad0b-9e65-8663-ecd0-3041ea6a19ba BootID:034926b2-e99f-4d20-933c-09ae42eb94e1 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f0:3d:e7:be:b1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f0:3d:e7:be:b1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:a0:af:1c:b9:7a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.992292 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.992436 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.995815 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.995843 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-68.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.996653 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.996665 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.996673 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.996689 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:40.998538 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.002155 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.002277 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.002306 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.002316 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.002329 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.002340 2572 kubelet.go:398] "Adding apiserver pod source" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.002350 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 09:06:41.247016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.002732 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.002750 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.005135 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.007484 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.007512 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009314 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009334 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009344 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009354 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009367 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009378 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009392 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009400 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009409 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009418 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009430 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.009438 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.010280 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.010292 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.015030 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-68.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.015080 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.015092 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.015374 2572 server.go:1268] "Started kubelet" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.015422 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.015472 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.015534 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.017574 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.017675 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.022035 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.023430 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-68.ec2.internal.18c04541b8d8e1ea default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-68.ec2.internal,UID:ip-10-0-136-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-68.ec2.internal,},FirstTimestamp:2026-07-08 09:06:41.015128554 +0000 UTC m=+0.427070483,LastTimestamp:2026-07-08 09:06:41.015128554 +0000 UTC m=+0.427070483,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-68.ec2.internal,}" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.025140 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.025151 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.025249 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-68.ec2.internal\" not found" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.025914 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.025932 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.025971 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.026024 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.026031 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.026462 2572 factory.go:153] Registering CRI-O factory Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.026477 2572 factory.go:223] Registration of the crio container factory successfully Jul 08 09:06:41.247950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.026525 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.026534 2572 factory.go:55] Registering systemd factory Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.026541 2572 factory.go:223] Registration of the systemd container factory successfully Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.026563 2572 factory.go:103] Registering Raw factory Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.026577 2572 manager.go:1196] Started watching for new ooms in manager Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.030969 2572 manager.go:349] Starting recovery of all containers Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.042057 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.042095 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.044364 2572 manager.go:354] Recovery completed Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.046225 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n6brs" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.048449 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.050273 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.050303 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.050318 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.050900 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.050909 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.050924 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.051648 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.052366 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.052392 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.052406 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.053440 2572 policy_none.go:49] "None policy: Start" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.053457 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.053470 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.054576 2572 policy_none.go:47] "Start" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.054736 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n6brs" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.083833 2572 manager.go:337] "Starting Device Plugin manager" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.083883 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.083894 2572 server.go:87] "Starting device plugin registration server" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.084153 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.084166 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.084349 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.084430 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.084440 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.084928 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.084983 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-68.ec2.internal\" not found" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.138568 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.139784 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.139818 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.139839 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.139870 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.142108 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.185046 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.186017 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.186049 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.186064 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.186091 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.369495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.198261 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.198281 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-68.ec2.internal\": node \"ip-10-0-136-68.ec2.internal\" not found" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.228711 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-68.ec2.internal\" not found" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.240697 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal"] Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.240796 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.242484 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.242510 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.242521 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.243722 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.243891 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.243931 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.244577 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.244591 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.244609 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.244609 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.244634 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.244619 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.245956 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.245986 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.247913 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.247949 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.247965 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.272365 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-68.ec2.internal\" not found" node="ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.276435 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-68.ec2.internal\" not found" node="ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.327698 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/802e55c5ab26814202be9eac69956172-config\") pod \"kube-apiserver-proxy-ip-10-0-136-68.ec2.internal\" (UID: \"802e55c5ab26814202be9eac69956172\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.327723 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8925c20d0f787a2bb875b826549cafa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal\" (UID: \"e8925c20d0f787a2bb875b826549cafa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.327743 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8925c20d0f787a2bb875b826549cafa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal\" (UID: \"e8925c20d0f787a2bb875b826549cafa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.370961 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.329787 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-68.ec2.internal\" not found" Jul 08 09:06:41.468339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.428088 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/802e55c5ab26814202be9eac69956172-config\") pod \"kube-apiserver-proxy-ip-10-0-136-68.ec2.internal\" (UID: \"802e55c5ab26814202be9eac69956172\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.468339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.428156 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8925c20d0f787a2bb875b826549cafa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal\" (UID: \"e8925c20d0f787a2bb875b826549cafa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.468339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.428114 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8925c20d0f787a2bb875b826549cafa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal\" (UID: \"e8925c20d0f787a2bb875b826549cafa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.468339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.428212 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/802e55c5ab26814202be9eac69956172-config\") pod \"kube-apiserver-proxy-ip-10-0-136-68.ec2.internal\" (UID: \"802e55c5ab26814202be9eac69956172\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.468339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.428221 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8925c20d0f787a2bb875b826549cafa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal\" (UID: \"e8925c20d0f787a2bb875b826549cafa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.468339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:41.428247 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8925c20d0f787a2bb875b826549cafa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal\" (UID: \"e8925c20d0f787a2bb875b826549cafa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" Jul 08 09:06:41.468339 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.430542 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-68.ec2.internal\" not found" Jul 08 09:06:42.753452 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:41.531245 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-68.ec2.internal\" not found" Jul 08 09:06:42.753452 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:42.737298 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" Jul 08 09:06:42.753452 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:42.743319 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal" Jul 08 09:06:42.862621 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:42.794164 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-68.ec2.internal\" not found" Jul 08 09:06:42.961835 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:42.902380 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-68.ec2.internal\" not found" Jul 08 09:06:43.078195 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.003109 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-68.ec2.internal\" not found" Jul 08 09:06:43.078195 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.048418 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.088130 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.099246 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.100858 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.116476 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.119602 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.119712 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.119764 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.164453 2572 apiserver.go:52] "Watching apiserver" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.180680 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.182031 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-65jmn","openshift-multus/multus-9vq6j","openshift-multus/network-metrics-daemon-lsqn9","openshift-network-diagnostics/network-check-target-g67tm","openshift-ovn-kubernetes/ovnkube-node-pw42g","kube-system/global-pull-secret-syncer-dwhwz","kube-system/konnectivity-agent-pbfjd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal","openshift-multus/multus-additional-cni-plugins-mfpzs","openshift-network-operator/iptables-alerter-bkll2","kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal","openshift-cluster-node-tuning-operator/tuned-297dp","openshift-dns/node-resolver-ht7f6"] Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.183556 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.184622 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.185902 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.185907 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.186088 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sdztk\"" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.186968 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wg45b\"" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.187022 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.187080 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.187223 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.193522 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.193569 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.194404 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.194441 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.195518 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198336 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-host\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198360 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:43.221742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198368 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef4d7d33-c0ca-4739-9c59-4b1c1b20a142-agent-certs\") pod \"konnectivity-agent-pbfjd\" (UID: \"ef4d7d33-c0ca-4739-9c59-4b1c1b20a142\") " pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198393 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef4d7d33-c0ca-4739-9c59-4b1c1b20a142-konnectivity-ca\") pod \"konnectivity-agent-pbfjd\" (UID: \"ef4d7d33-c0ca-4739-9c59-4b1c1b20a142\") " pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.198421 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198451 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-sysconfig\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198491 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-kubernetes\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198514 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-sysctl-d\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198543 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-sysctl-conf\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198582 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-var-lib-kubelet\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198615 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198617 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-tmp\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198674 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-modprobe-d\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198687 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198697 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-systemd\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198704 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mb8ts\"" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198728 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-run\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198764 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-lib-modules\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198805 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198807 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vfdn\" (UniqueName: \"kubernetes.io/projected/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-kube-api-access-2vfdn\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198812 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 09:06:43.223256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198831 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198868 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2nvd\" (UniqueName: \"kubernetes.io/projected/b9eeaebc-02be-4f69-b34d-6f8135fc5374-kube-api-access-q2nvd\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198900 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-sys\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.198923 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-tuned\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.199029 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.199059 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.199087 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.199569 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.200777 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.201883 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.203205 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.204602 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.204763 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.204879 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.204980 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.205073 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.205244 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.205275 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.205576 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.205854 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4x8v9\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.205928 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mt7gs\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.206077 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.206233 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.206327 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ps99l\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.206333 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.206589 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-b2d89\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.206608 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.206589 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.208813 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.208955 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.209079 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.209136 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ch98w\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.209153 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zn6wp\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.209155 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 09:06:43.223888 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.209158 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 09:06:43.224855 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.210023 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 09:06:43.224855 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.219308 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 09:01:42 +0000 UTC" deadline="2028-03-05 09:41:07.192896419 +0000 UTC" Jul 08 09:06:43.224855 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.219346 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14544h34m23.973553177s" Jul 08 09:06:43.224855 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.223023 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-q4ntc" Jul 08 09:06:43.231811 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.231797 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q4ntc" Jul 08 09:06:43.289122 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.289100 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 09:06:43.299130 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299100 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t6xh\" (UniqueName: \"kubernetes.io/projected/3b97acfe-ed2a-415d-9223-746ddcdb5115-kube-api-access-7t6xh\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.299215 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299134 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-node-log\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.299215 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299153 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh64h\" (UniqueName: \"kubernetes.io/projected/61ca3fd1-d587-4a48-a64e-d2eaf482dc92-kube-api-access-hh64h\") pod \"node-ca-65jmn\" (UID: \"61ca3fd1-d587-4a48-a64e-d2eaf482dc92\") " pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.299282 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299206 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b3d1f87-66be-4e09-99c1-c1fa7b7ba552-host-slash\") pod \"iptables-alerter-bkll2\" (UID: \"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552\") " pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.299282 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299240 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-device-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.299282 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299272 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-systemd-units\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.299417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299291 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-slash\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.299417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299309 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-host\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.299417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299324 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8f8k\" (UniqueName: \"kubernetes.io/projected/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-kube-api-access-w8f8k\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.299417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299357 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-cni-netd\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.299417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299399 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-host\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.299638 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299437 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.299638 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299471 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2457b473-47d8-4489-9ef2-832c2aaaedd1-ovnkube-config\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.299638 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299499 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2457b473-47d8-4489-9ef2-832c2aaaedd1-env-overrides\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.299638 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299529 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-sysctl-conf\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.299638 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299552 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-tmp\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.299638 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299576 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-system-cni-dir\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.299638 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299598 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61ca3fd1-d587-4a48-a64e-d2eaf482dc92-serviceca\") pod \"node-ca-65jmn\" (UID: \"61ca3fd1-d587-4a48-a64e-d2eaf482dc92\") " pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.299638 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299625 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-kubelet\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.299900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299643 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-sysctl-conf\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.299900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299648 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-tuned\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.299900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299687 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.299900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299707 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-socket-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.299900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299722 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-registration-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.299900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299778 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-os-release\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.299900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299808 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-socket-dir-parent\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.299900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299833 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-etc-kubernetes\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.299900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299857 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2hrr\" (UniqueName: \"kubernetes.io/projected/a6c58c8c-52bc-4924-92f2-13063e6bc4de-kube-api-access-p2hrr\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.299900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299881 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-run-netns\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299903 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-var-lib-openvswitch\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299925 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-etc-openvswitch\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299947 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-cni-bin\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.299985 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-kubernetes\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300010 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a6c58c8c-52bc-4924-92f2-13063e6bc4de-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300034 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300033 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-kubernetes\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300066 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300097 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/048d9fac-8d08-46a7-a4c6-bae6f85821b6-tmp-dir\") pod \"node-resolver-ht7f6\" (UID: \"048d9fac-8d08-46a7-a4c6-bae6f85821b6\") " pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300121 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-run-netns\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300139 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-run-multus-certs\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.300158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300155 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-system-cni-dir\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300171 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-cnibin\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300196 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-cni-dir\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300213 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-run-k8s-cni-cncf-io\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300226 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-hostroot\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300244 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2457b473-47d8-4489-9ef2-832c2aaaedd1-ovnkube-script-lib\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300271 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c8c7dd60-3fc1-45b5-baae-d2867dd09577-dbus\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300290 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b97acfe-ed2a-415d-9223-746ddcdb5115-cni-binary-copy\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300306 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a6c58c8c-52bc-4924-92f2-13063e6bc4de-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300343 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-run-systemd\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300361 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-run-ovn\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300387 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbkwd\" (UniqueName: \"kubernetes.io/projected/2457b473-47d8-4489-9ef2-832c2aaaedd1-kube-api-access-fbkwd\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300407 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/048d9fac-8d08-46a7-a4c6-bae6f85821b6-hosts-file\") pod \"node-resolver-ht7f6\" (UID: \"048d9fac-8d08-46a7-a4c6-bae6f85821b6\") " pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300421 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61ca3fd1-d587-4a48-a64e-d2eaf482dc92-host\") pod \"node-ca-65jmn\" (UID: \"61ca3fd1-d587-4a48-a64e-d2eaf482dc92\") " pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.300464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300453 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef4d7d33-c0ca-4739-9c59-4b1c1b20a142-konnectivity-ca\") pod \"konnectivity-agent-pbfjd\" (UID: \"ef4d7d33-c0ca-4739-9c59-4b1c1b20a142\") " pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300483 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-sysconfig\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300502 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-sysctl-d\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300521 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-var-lib-kubelet\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300538 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-sysconfig\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300562 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vfdn\" (UniqueName: \"kubernetes.io/projected/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-kube-api-access-2vfdn\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300586 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-var-lib-cni-bin\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300606 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6c58c8c-52bc-4924-92f2-13063e6bc4de-cni-binary-copy\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300607 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-var-lib-kubelet\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300702 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-sysctl-d\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300725 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-run-openvswitch\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300743 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-modprobe-d\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300774 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-systemd\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300796 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-run\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300812 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-lib-modules\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300827 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300843 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-conf-dir\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.300890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300858 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-systemd\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300864 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-modprobe-d\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300865 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-os-release\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300918 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-lib-modules\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300920 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-run\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300956 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.300970 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300978 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-sys\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.300993 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-var-lib-kubelet\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301016 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9b3d1f87-66be-4e09-99c1-c1fa7b7ba552-iptables-alerter-script\") pod \"iptables-alerter-bkll2\" (UID: \"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552\") " pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.301036 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs podName:b9eeaebc-02be-4f69-b34d-6f8135fc5374 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:43.801020849 +0000 UTC m=+2.050980114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs") pod "network-metrics-daemon-lsqn9" (UID: "b9eeaebc-02be-4f69-b34d-6f8135fc5374") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301047 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-sys\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301054 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef4d7d33-c0ca-4739-9c59-4b1c1b20a142-konnectivity-ca\") pod \"konnectivity-agent-pbfjd\" (UID: \"ef4d7d33-c0ca-4739-9c59-4b1c1b20a142\") " pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301065 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2457b473-47d8-4489-9ef2-832c2aaaedd1-ovn-node-metrics-cert\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301114 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c8c7dd60-3fc1-45b5-baae-d2867dd09577-kubelet-config\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301142 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-cnibin\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301173 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-var-lib-cni-multus\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.301469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301199 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-daemon-config\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.302078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301248 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-sys-fs\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.302078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301269 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxbw6\" (UniqueName: \"kubernetes.io/projected/048d9fac-8d08-46a7-a4c6-bae6f85821b6-kube-api-access-fxbw6\") pod \"node-resolver-ht7f6\" (UID: \"048d9fac-8d08-46a7-a4c6-bae6f85821b6\") " pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.302078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301295 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef4d7d33-c0ca-4739-9c59-4b1c1b20a142-agent-certs\") pod \"konnectivity-agent-pbfjd\" (UID: \"ef4d7d33-c0ca-4739-9c59-4b1c1b20a142\") " pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:06:43.302078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301350 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2nvd\" (UniqueName: \"kubernetes.io/projected/b9eeaebc-02be-4f69-b34d-6f8135fc5374-kube-api-access-q2nvd\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:43.302078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301401 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq62t\" (UniqueName: \"kubernetes.io/projected/9b3d1f87-66be-4e09-99c1-c1fa7b7ba552-kube-api-access-jq62t\") pod \"iptables-alerter-bkll2\" (UID: \"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552\") " pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.302078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301420 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxx28\" (UniqueName: \"kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28\") pod \"network-check-target-g67tm\" (UID: \"071ff18b-ef9e-46ef-a714-e167ad2b30d7\") " pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:43.302078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301443 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-log-socket\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.302078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301464 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-run-ovn-kubernetes\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.302078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.301475 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 09:06:43.304501 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.304484 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-etc-tuned\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.304618 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.304500 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-tmp\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.304710 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.304695 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef4d7d33-c0ca-4739-9c59-4b1c1b20a142-agent-certs\") pod \"konnectivity-agent-pbfjd\" (UID: \"ef4d7d33-c0ca-4739-9c59-4b1c1b20a142\") " pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:06:43.309586 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.309558 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vfdn\" (UniqueName: \"kubernetes.io/projected/2d1e824a-40d2-48f9-8ca1-e8f69f686beb-kube-api-access-2vfdn\") pod \"tuned-297dp\" (UID: \"2d1e824a-40d2-48f9-8ca1-e8f69f686beb\") " pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.311803 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.311742 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2nvd\" (UniqueName: \"kubernetes.io/projected/b9eeaebc-02be-4f69-b34d-6f8135fc5374-kube-api-access-q2nvd\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:43.401742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401714 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-run-ovn\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.401932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401763 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkwd\" (UniqueName: \"kubernetes.io/projected/2457b473-47d8-4489-9ef2-832c2aaaedd1-kube-api-access-fbkwd\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.401932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401790 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/048d9fac-8d08-46a7-a4c6-bae6f85821b6-hosts-file\") pod \"node-resolver-ht7f6\" (UID: \"048d9fac-8d08-46a7-a4c6-bae6f85821b6\") " pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.401932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401811 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61ca3fd1-d587-4a48-a64e-d2eaf482dc92-host\") pod \"node-ca-65jmn\" (UID: \"61ca3fd1-d587-4a48-a64e-d2eaf482dc92\") " pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.401932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401829 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-run-ovn\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.401932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401838 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-var-lib-cni-bin\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.401932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401861 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6c58c8c-52bc-4924-92f2-13063e6bc4de-cni-binary-copy\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.401932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401885 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-run-openvswitch\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.401932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401889 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61ca3fd1-d587-4a48-a64e-d2eaf482dc92-host\") pod \"node-ca-65jmn\" (UID: \"61ca3fd1-d587-4a48-a64e-d2eaf482dc92\") " pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.401932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401914 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-var-lib-cni-bin\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401941 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/048d9fac-8d08-46a7-a4c6-bae6f85821b6-hosts-file\") pod \"node-resolver-ht7f6\" (UID: \"048d9fac-8d08-46a7-a4c6-bae6f85821b6\") " pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401924 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-conf-dir\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401960 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-conf-dir\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401977 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-run-openvswitch\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.401980 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-os-release\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402017 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402028 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-os-release\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402045 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-var-lib-kubelet\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402084 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-var-lib-kubelet\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402101 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9b3d1f87-66be-4e09-99c1-c1fa7b7ba552-iptables-alerter-script\") pod \"iptables-alerter-bkll2\" (UID: \"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552\") " pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402143 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2457b473-47d8-4489-9ef2-832c2aaaedd1-ovn-node-metrics-cert\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c8c7dd60-3fc1-45b5-baae-d2867dd09577-kubelet-config\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402184 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-cnibin\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402214 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c8c7dd60-3fc1-45b5-baae-d2867dd09577-kubelet-config\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402237 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-cnibin\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402219 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.402306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402257 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-var-lib-cni-multus\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402281 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-daemon-config\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402285 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-var-lib-cni-multus\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402303 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-sys-fs\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402328 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxbw6\" (UniqueName: \"kubernetes.io/projected/048d9fac-8d08-46a7-a4c6-bae6f85821b6-kube-api-access-fxbw6\") pod \"node-resolver-ht7f6\" (UID: \"048d9fac-8d08-46a7-a4c6-bae6f85821b6\") " pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402354 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq62t\" (UniqueName: \"kubernetes.io/projected/9b3d1f87-66be-4e09-99c1-c1fa7b7ba552-kube-api-access-jq62t\") pod \"iptables-alerter-bkll2\" (UID: \"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552\") " pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402379 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx28\" (UniqueName: \"kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28\") pod \"network-check-target-g67tm\" (UID: \"071ff18b-ef9e-46ef-a714-e167ad2b30d7\") " pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402403 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-log-socket\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402412 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-sys-fs\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-run-ovn-kubernetes\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402463 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t6xh\" (UniqueName: \"kubernetes.io/projected/3b97acfe-ed2a-415d-9223-746ddcdb5115-kube-api-access-7t6xh\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402493 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-node-log\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402536 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-node-log\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402466 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-run-ovn-kubernetes\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402692 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh64h\" (UniqueName: \"kubernetes.io/projected/61ca3fd1-d587-4a48-a64e-d2eaf482dc92-kube-api-access-hh64h\") pod \"node-ca-65jmn\" (UID: \"61ca3fd1-d587-4a48-a64e-d2eaf482dc92\") " pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402722 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b3d1f87-66be-4e09-99c1-c1fa7b7ba552-host-slash\") pod \"iptables-alerter-bkll2\" (UID: \"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552\") " pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402732 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9b3d1f87-66be-4e09-99c1-c1fa7b7ba552-iptables-alerter-script\") pod \"iptables-alerter-bkll2\" (UID: \"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552\") " pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.403120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402765 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-device-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402792 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-systemd-units\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402808 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-daemon-config\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402816 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-slash\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402827 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-log-socket\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402864 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8f8k\" (UniqueName: \"kubernetes.io/projected/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-kube-api-access-w8f8k\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402869 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b3d1f87-66be-4e09-99c1-c1fa7b7ba552-host-slash\") pod \"iptables-alerter-bkll2\" (UID: \"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552\") " pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402875 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-systemd-units\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402873 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-device-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402895 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-cni-netd\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402910 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-slash\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402932 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402936 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-cni-netd\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.402981 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403003 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2457b473-47d8-4489-9ef2-832c2aaaedd1-ovnkube-config\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403027 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2457b473-47d8-4489-9ef2-832c2aaaedd1-env-overrides\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403111 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-system-cni-dir\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.403926 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403157 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61ca3fd1-d587-4a48-a64e-d2eaf482dc92-serviceca\") pod \"node-ca-65jmn\" (UID: \"61ca3fd1-d587-4a48-a64e-d2eaf482dc92\") " pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403185 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-kubelet\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403210 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403234 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-socket-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403259 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-registration-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403263 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-kubelet\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403286 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-os-release\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403297 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403296 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6c58c8c-52bc-4924-92f2-13063e6bc4de-cni-binary-copy\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403312 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-socket-dir-parent\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403350 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-etc-kubernetes\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403370 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-socket-dir-parent\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403377 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hrr\" (UniqueName: \"kubernetes.io/projected/a6c58c8c-52bc-4924-92f2-13063e6bc4de-kube-api-access-p2hrr\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403261 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-system-cni-dir\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403404 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-run-netns\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403430 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-var-lib-openvswitch\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403444 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-registration-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.404691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403453 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-etc-openvswitch\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403446 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-os-release\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403404 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-socket-dir\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403478 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-etc-kubernetes\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403515 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-var-lib-openvswitch\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403517 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-run-netns\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403546 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2457b473-47d8-4489-9ef2-832c2aaaedd1-env-overrides\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403550 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-etc-openvswitch\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403604 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-cni-bin\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403634 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a6c58c8c-52bc-4924-92f2-13063e6bc4de-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403663 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-host-cni-bin\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403671 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403698 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403711 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61ca3fd1-d587-4a48-a64e-d2eaf482dc92-serviceca\") pod \"node-ca-65jmn\" (UID: \"61ca3fd1-d587-4a48-a64e-d2eaf482dc92\") " pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403722 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/048d9fac-8d08-46a7-a4c6-bae6f85821b6-tmp-dir\") pod \"node-resolver-ht7f6\" (UID: \"048d9fac-8d08-46a7-a4c6-bae6f85821b6\") " pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403744 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-run-netns\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403772 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.405207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403785 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-run-multus-certs\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403808 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-system-cni-dir\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.403812 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403818 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-run-netns\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403831 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-cnibin\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403857 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-cni-dir\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403879 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-run-k8s-cni-cncf-io\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.403895 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret podName:c8c7dd60-3fc1-45b5-baae-d2867dd09577 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:43.903875472 +0000 UTC m=+2.153834744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret") pod "global-pull-secret-syncer-dwhwz" (UID: "c8c7dd60-3fc1-45b5-baae-d2867dd09577") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403927 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-hostroot\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403931 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-cnibin\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403954 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2457b473-47d8-4489-9ef2-832c2aaaedd1-ovnkube-script-lib\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403975 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6c58c8c-52bc-4924-92f2-13063e6bc4de-system-cni-dir\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403978 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c8c7dd60-3fc1-45b5-baae-d2867dd09577-dbus\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404003 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b97acfe-ed2a-415d-9223-746ddcdb5115-cni-binary-copy\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404026 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-hostroot\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404028 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a6c58c8c-52bc-4924-92f2-13063e6bc4de-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404045 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/048d9fac-8d08-46a7-a4c6-bae6f85821b6-tmp-dir\") pod \"node-resolver-ht7f6\" (UID: \"048d9fac-8d08-46a7-a4c6-bae6f85821b6\") " pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.405666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404056 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-multus-cni-dir\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404051 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-run-systemd\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404078 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2457b473-47d8-4489-9ef2-832c2aaaedd1-run-systemd\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404112 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c8c7dd60-3fc1-45b5-baae-d2867dd09577-dbus\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403928 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-run-k8s-cni-cncf-io\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404146 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a6c58c8c-52bc-4924-92f2-13063e6bc4de-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.403858 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3b97acfe-ed2a-415d-9223-746ddcdb5115-host-run-multus-certs\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404554 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a6c58c8c-52bc-4924-92f2-13063e6bc4de-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404590 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2457b473-47d8-4489-9ef2-832c2aaaedd1-ovnkube-config\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404622 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b97acfe-ed2a-415d-9223-746ddcdb5115-cni-binary-copy\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404675 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2457b473-47d8-4489-9ef2-832c2aaaedd1-ovnkube-script-lib\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.406165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.404813 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2457b473-47d8-4489-9ef2-832c2aaaedd1-ovn-node-metrics-cert\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.408832 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.408809 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:06:43.408931 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.408835 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:06:43.408931 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.408848 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxx28 for pod openshift-network-diagnostics/network-check-target-g67tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:43.408931 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.408913 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28 podName:071ff18b-ef9e-46ef-a714-e167ad2b30d7 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:43.908895959 +0000 UTC m=+2.158855221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sxx28" (UniqueName: "kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28") pod "network-check-target-g67tm" (UID: "071ff18b-ef9e-46ef-a714-e167ad2b30d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:43.411222 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.411201 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxbw6\" (UniqueName: \"kubernetes.io/projected/048d9fac-8d08-46a7-a4c6-bae6f85821b6-kube-api-access-fxbw6\") pod \"node-resolver-ht7f6\" (UID: \"048d9fac-8d08-46a7-a4c6-bae6f85821b6\") " pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.411447 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.411415 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8f8k\" (UniqueName: \"kubernetes.io/projected/d3db75a5-55ba-4be6-99f9-eb9712ff28c7-kube-api-access-w8f8k\") pod \"aws-ebs-csi-driver-node-qgrnt\" (UID: \"d3db75a5-55ba-4be6-99f9-eb9712ff28c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.411535 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.411505 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbkwd\" (UniqueName: \"kubernetes.io/projected/2457b473-47d8-4489-9ef2-832c2aaaedd1-kube-api-access-fbkwd\") pod \"ovnkube-node-pw42g\" (UID: \"2457b473-47d8-4489-9ef2-832c2aaaedd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.413359 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.413332 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh64h\" (UniqueName: \"kubernetes.io/projected/61ca3fd1-d587-4a48-a64e-d2eaf482dc92-kube-api-access-hh64h\") pod \"node-ca-65jmn\" (UID: \"61ca3fd1-d587-4a48-a64e-d2eaf482dc92\") " pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.413448 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.413418 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq62t\" (UniqueName: \"kubernetes.io/projected/9b3d1f87-66be-4e09-99c1-c1fa7b7ba552-kube-api-access-jq62t\") pod \"iptables-alerter-bkll2\" (UID: \"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552\") " pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.413644 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.413629 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t6xh\" (UniqueName: \"kubernetes.io/projected/3b97acfe-ed2a-415d-9223-746ddcdb5115-kube-api-access-7t6xh\") pod \"multus-9vq6j\" (UID: \"3b97acfe-ed2a-415d-9223-746ddcdb5115\") " pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.415127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.415101 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2hrr\" (UniqueName: \"kubernetes.io/projected/a6c58c8c-52bc-4924-92f2-13063e6bc4de-kube-api-access-p2hrr\") pod \"multus-additional-cni-plugins-mfpzs\" (UID: \"a6c58c8c-52bc-4924-92f2-13063e6bc4de\") " pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.507462 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:43.507421 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8925c20d0f787a2bb875b826549cafa.slice/crio-d6fbf2e534c1fe89574fe549b3d84296a46aeee60d3fb5f12343b06be0477f28 WatchSource:0}: Error finding container d6fbf2e534c1fe89574fe549b3d84296a46aeee60d3fb5f12343b06be0477f28: Status 404 returned error can't find the container with id d6fbf2e534c1fe89574fe549b3d84296a46aeee60d3fb5f12343b06be0477f28 Jul 08 09:06:43.508095 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:43.508074 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod802e55c5ab26814202be9eac69956172.slice/crio-2ab18de594f6b03c840acd164dd149f536d267b11832ef57bbe7be479e271cd5 WatchSource:0}: Error finding container 2ab18de594f6b03c840acd164dd149f536d267b11832ef57bbe7be479e271cd5: Status 404 returned error can't find the container with id 2ab18de594f6b03c840acd164dd149f536d267b11832ef57bbe7be479e271cd5 Jul 08 09:06:43.512555 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.512538 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 09:06:43.520383 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.520364 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:06:43.529367 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:43.529346 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef4d7d33_c0ca_4739_9c59_4b1c1b20a142.slice/crio-f143b15ddd2da33ce7ede91fe31eafb4b8d45232f501736200ab3201fb988ff6 WatchSource:0}: Error finding container f143b15ddd2da33ce7ede91fe31eafb4b8d45232f501736200ab3201fb988ff6: Status 404 returned error can't find the container with id f143b15ddd2da33ce7ede91fe31eafb4b8d45232f501736200ab3201fb988ff6 Jul 08 09:06:43.541764 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.541723 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-297dp" Jul 08 09:06:43.549762 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:43.549729 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1e824a_40d2_48f9_8ca1_e8f69f686beb.slice/crio-6c9fbb02bb51209aa821d42bcb8f4ae5ca6c8e346c17d4f789817a6e5e22e7ff WatchSource:0}: Error finding container 6c9fbb02bb51209aa821d42bcb8f4ae5ca6c8e346c17d4f789817a6e5e22e7ff: Status 404 returned error can't find the container with id 6c9fbb02bb51209aa821d42bcb8f4ae5ca6c8e346c17d4f789817a6e5e22e7ff Jul 08 09:06:43.573900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.573877 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:06:43.583092 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:43.583065 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2457b473_47d8_4489_9ef2_832c2aaaedd1.slice/crio-1ecd3a974c09d6f9e25459b42d164787a94750b9fc80bc209472ede86d363a10 WatchSource:0}: Error finding container 1ecd3a974c09d6f9e25459b42d164787a94750b9fc80bc209472ede86d363a10: Status 404 returned error can't find the container with id 1ecd3a974c09d6f9e25459b42d164787a94750b9fc80bc209472ede86d363a10 Jul 08 09:06:43.590060 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.590044 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ht7f6" Jul 08 09:06:43.591126 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.591106 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-65jmn" Jul 08 09:06:43.598439 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:43.598417 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048d9fac_8d08_46a7_a4c6_bae6f85821b6.slice/crio-5e1480bc407bbd2df04bd8fcf9e7d1c5a16a725ac7d77e7fabe91a6267c21dcf WatchSource:0}: Error finding container 5e1480bc407bbd2df04bd8fcf9e7d1c5a16a725ac7d77e7fabe91a6267c21dcf: Status 404 returned error can't find the container with id 5e1480bc407bbd2df04bd8fcf9e7d1c5a16a725ac7d77e7fabe91a6267c21dcf Jul 08 09:06:43.616878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.616853 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9vq6j" Jul 08 09:06:43.617314 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.617296 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:06:43.626006 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:43.625984 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b97acfe_ed2a_415d_9223_746ddcdb5115.slice/crio-23afcfd2811d9cba33cb6673451481b82b6b98e0e1d1fcf36aa47ba22b11bb33 WatchSource:0}: Error finding container 23afcfd2811d9cba33cb6673451481b82b6b98e0e1d1fcf36aa47ba22b11bb33: Status 404 returned error can't find the container with id 23afcfd2811d9cba33cb6673451481b82b6b98e0e1d1fcf36aa47ba22b11bb33 Jul 08 09:06:43.638562 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.638544 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" Jul 08 09:06:43.644475 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.644456 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" Jul 08 09:06:43.646640 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:43.646616 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6c58c8c_52bc_4924_92f2_13063e6bc4de.slice/crio-982b6cc65d2f7f4b1d43d5ccf21abc827f0b74ed155afd2b8dd90f435f453135 WatchSource:0}: Error finding container 982b6cc65d2f7f4b1d43d5ccf21abc827f0b74ed155afd2b8dd90f435f453135: Status 404 returned error can't find the container with id 982b6cc65d2f7f4b1d43d5ccf21abc827f0b74ed155afd2b8dd90f435f453135 Jul 08 09:06:43.653010 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:43.652984 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3db75a5_55ba_4be6_99f9_eb9712ff28c7.slice/crio-85777640067da366f07be1aafda1c30722034f6290cf943e933ea041165b9784 WatchSource:0}: Error finding container 85777640067da366f07be1aafda1c30722034f6290cf943e933ea041165b9784: Status 404 returned error can't find the container with id 85777640067da366f07be1aafda1c30722034f6290cf943e933ea041165b9784 Jul 08 09:06:43.663816 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.663800 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bkll2" Jul 08 09:06:43.672677 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:06:43.672658 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3d1f87_66be_4e09_99c1_c1fa7b7ba552.slice/crio-2a62c5d2e2cbacd590857a6b3405f4930afe6265345439f2eb1de0df591f475c WatchSource:0}: Error finding container 2a62c5d2e2cbacd590857a6b3405f4930afe6265345439f2eb1de0df591f475c: Status 404 returned error can't find the container with id 2a62c5d2e2cbacd590857a6b3405f4930afe6265345439f2eb1de0df591f475c Jul 08 09:06:43.674711 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.674693 2572 kuberuntime_manager.go:1459] "Unhandled Error" err="container iptables-alerter start failed in pod iptables-alerter-bkll2_openshift-network-operator(9b3d1f87-66be-4e09-99c1-c1fa7b7ba552): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 08 09:06:43.674787 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.674724 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-bkll2" podUID="9b3d1f87-66be-4e09-99c1-c1fa7b7ba552" Jul 08 09:06:43.799474 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.799389 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:06:43.807932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.807909 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:43.808071 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.808056 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:43.808119 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.808110 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs podName:b9eeaebc-02be-4f69-b34d-6f8135fc5374 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:44.808096525 +0000 UTC m=+3.058055792 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs") pod "network-metrics-daemon-lsqn9" (UID: "b9eeaebc-02be-4f69-b34d-6f8135fc5374") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:43.908935 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.908901 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:43.909071 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:43.908958 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx28\" (UniqueName: \"kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28\") pod \"network-check-target-g67tm\" (UID: \"071ff18b-ef9e-46ef-a714-e167ad2b30d7\") " pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:43.909071 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.909052 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:43.909171 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.909087 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:06:43.909171 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.909111 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:06:43.909171 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.909121 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxx28 for pod openshift-network-diagnostics/network-check-target-g67tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:43.909171 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.909122 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret podName:c8c7dd60-3fc1-45b5-baae-d2867dd09577 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:44.909107491 +0000 UTC m=+3.159066756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret") pod "global-pull-secret-syncer-dwhwz" (UID: "c8c7dd60-3fc1-45b5-baae-d2867dd09577") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:43.909171 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:43.909164 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28 podName:071ff18b-ef9e-46ef-a714-e167ad2b30d7 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:44.909153559 +0000 UTC m=+3.159112820 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxx28" (UniqueName: "kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28") pod "network-check-target-g67tm" (UID: "071ff18b-ef9e-46ef-a714-e167ad2b30d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:44.032349 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.032311 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:06:44.233399 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.233284 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 09:01:43 +0000 UTC" deadline="2028-02-16 15:13:47.301531247 +0000 UTC" Jul 08 09:06:44.233399 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.233353 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14118h7m3.068183111s" Jul 08 09:06:44.332922 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.332859 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" event={"ID":"d3db75a5-55ba-4be6-99f9-eb9712ff28c7","Type":"ContainerStarted","Data":"85777640067da366f07be1aafda1c30722034f6290cf943e933ea041165b9784"} Jul 08 09:06:44.341301 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.341261 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" event={"ID":"a6c58c8c-52bc-4924-92f2-13063e6bc4de","Type":"ContainerStarted","Data":"982b6cc65d2f7f4b1d43d5ccf21abc827f0b74ed155afd2b8dd90f435f453135"} Jul 08 09:06:44.344764 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.344723 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9vq6j" event={"ID":"3b97acfe-ed2a-415d-9223-746ddcdb5115","Type":"ContainerStarted","Data":"23afcfd2811d9cba33cb6673451481b82b6b98e0e1d1fcf36aa47ba22b11bb33"} Jul 08 09:06:44.349340 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.349241 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" event={"ID":"2457b473-47d8-4489-9ef2-832c2aaaedd1","Type":"ContainerStarted","Data":"1ecd3a974c09d6f9e25459b42d164787a94750b9fc80bc209472ede86d363a10"} Jul 08 09:06:44.353965 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.353679 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bkll2" event={"ID":"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552","Type":"ContainerStarted","Data":"2a62c5d2e2cbacd590857a6b3405f4930afe6265345439f2eb1de0df591f475c"} Jul 08 09:06:44.359345 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:44.358807 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:28389dd118629aa9d7c2525202f3ad6359afb98b6a3bf7672cb1ecfae2418e23\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-bkll2" podUID="9b3d1f87-66be-4e09-99c1-c1fa7b7ba552" Jul 08 09:06:44.359345 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.359323 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-65jmn" event={"ID":"61ca3fd1-d587-4a48-a64e-d2eaf482dc92","Type":"ContainerStarted","Data":"f6549a00eae0e8cac7941f7e6f5ea1a0718f29b2b2cc394a3be31c9c1f310975"} Jul 08 09:06:44.362642 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.362163 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ht7f6" event={"ID":"048d9fac-8d08-46a7-a4c6-bae6f85821b6","Type":"ContainerStarted","Data":"5e1480bc407bbd2df04bd8fcf9e7d1c5a16a725ac7d77e7fabe91a6267c21dcf"} Jul 08 09:06:44.363474 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.363450 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-297dp" event={"ID":"2d1e824a-40d2-48f9-8ca1-e8f69f686beb","Type":"ContainerStarted","Data":"6c9fbb02bb51209aa821d42bcb8f4ae5ca6c8e346c17d4f789817a6e5e22e7ff"} Jul 08 09:06:44.364802 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.364779 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pbfjd" event={"ID":"ef4d7d33-c0ca-4739-9c59-4b1c1b20a142","Type":"ContainerStarted","Data":"f143b15ddd2da33ce7ede91fe31eafb4b8d45232f501736200ab3201fb988ff6"} Jul 08 09:06:44.368230 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.368173 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal" event={"ID":"802e55c5ab26814202be9eac69956172","Type":"ContainerStarted","Data":"2ab18de594f6b03c840acd164dd149f536d267b11832ef57bbe7be479e271cd5"} Jul 08 09:06:44.371954 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.371930 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" event={"ID":"e8925c20d0f787a2bb875b826549cafa","Type":"ContainerStarted","Data":"d6fbf2e534c1fe89574fe549b3d84296a46aeee60d3fb5f12343b06be0477f28"} Jul 08 09:06:44.824150 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.824112 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:44.824335 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:44.824266 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:44.824335 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:44.824329 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs podName:b9eeaebc-02be-4f69-b34d-6f8135fc5374 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:46.824308602 +0000 UTC m=+5.074267880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs") pod "network-metrics-daemon-lsqn9" (UID: "b9eeaebc-02be-4f69-b34d-6f8135fc5374") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:44.925052 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.925010 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx28\" (UniqueName: \"kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28\") pod \"network-check-target-g67tm\" (UID: \"071ff18b-ef9e-46ef-a714-e167ad2b30d7\") " pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:44.925240 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:44.925094 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:44.925240 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:44.925223 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:44.925351 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:44.925284 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret podName:c8c7dd60-3fc1-45b5-baae-d2867dd09577 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:46.92526608 +0000 UTC m=+5.175225360 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret") pod "global-pull-secret-syncer-dwhwz" (UID: "c8c7dd60-3fc1-45b5-baae-d2867dd09577") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:44.925351 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:44.925283 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:06:44.925351 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:44.925311 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:06:44.925351 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:44.925325 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxx28 for pod openshift-network-diagnostics/network-check-target-g67tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:44.925535 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:44.925377 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28 podName:071ff18b-ef9e-46ef-a714-e167ad2b30d7 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:46.925360468 +0000 UTC m=+5.175319732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxx28" (UniqueName: "kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28") pod "network-check-target-g67tm" (UID: "071ff18b-ef9e-46ef-a714-e167ad2b30d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:45.234226 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:45.234106 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 09:01:43 +0000 UTC" deadline="2028-02-05 14:28:26.246240894 +0000 UTC" Jul 08 09:06:45.234226 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:45.234149 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13853h21m41.01209721s" Jul 08 09:06:45.254891 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:45.254641 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:06:45.302469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:45.302437 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:45.302651 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:45.302576 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:06:45.303376 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:45.302871 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:45.303376 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:45.302982 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:06:45.303376 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:45.303147 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:45.303376 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:45.303235 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:06:46.842344 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:46.842292 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:46.842976 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:46.842478 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:46.842976 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:46.842622 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs podName:b9eeaebc-02be-4f69-b34d-6f8135fc5374 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:50.842596915 +0000 UTC m=+9.092556199 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs") pod "network-metrics-daemon-lsqn9" (UID: "b9eeaebc-02be-4f69-b34d-6f8135fc5374") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:46.943273 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:46.943237 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:46.943470 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:46.943325 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx28\" (UniqueName: \"kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28\") pod \"network-check-target-g67tm\" (UID: \"071ff18b-ef9e-46ef-a714-e167ad2b30d7\") " pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:46.943470 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:46.943388 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:46.943470 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:46.943464 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret podName:c8c7dd60-3fc1-45b5-baae-d2867dd09577 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:50.943444732 +0000 UTC m=+9.193403996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret") pod "global-pull-secret-syncer-dwhwz" (UID: "c8c7dd60-3fc1-45b5-baae-d2867dd09577") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:46.943698 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:46.943477 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:06:46.943698 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:46.943496 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:06:46.943698 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:46.943508 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxx28 for pod openshift-network-diagnostics/network-check-target-g67tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:46.943698 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:46.943602 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28 podName:071ff18b-ef9e-46ef-a714-e167ad2b30d7 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:50.94357411 +0000 UTC m=+9.193533394 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxx28" (UniqueName: "kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28") pod "network-check-target-g67tm" (UID: "071ff18b-ef9e-46ef-a714-e167ad2b30d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:47.302430 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:47.302351 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:47.302723 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:47.302487 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:06:47.302723 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:47.302647 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:47.303132 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:47.302790 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:06:47.303132 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:47.302828 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:47.303132 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:47.302927 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:06:49.302363 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:49.302326 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:49.302966 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:49.302344 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:49.302966 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:49.302459 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:49.302966 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:49.302461 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:06:49.302966 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:49.302579 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:06:49.302966 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:49.302695 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:06:50.877563 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:50.877309 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:50.877563 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:50.877476 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:50.877563 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:50.877541 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs podName:b9eeaebc-02be-4f69-b34d-6f8135fc5374 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:58.877521701 +0000 UTC m=+17.127480974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs") pod "network-metrics-daemon-lsqn9" (UID: "b9eeaebc-02be-4f69-b34d-6f8135fc5374") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:50.978557 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:50.978468 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx28\" (UniqueName: \"kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28\") pod \"network-check-target-g67tm\" (UID: \"071ff18b-ef9e-46ef-a714-e167ad2b30d7\") " pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:50.978557 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:50.978537 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:50.978905 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:50.978654 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:06:50.978905 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:50.978683 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:06:50.978905 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:50.978680 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:50.978905 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:50.978697 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxx28 for pod openshift-network-diagnostics/network-check-target-g67tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:50.978905 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:50.978769 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret podName:c8c7dd60-3fc1-45b5-baae-d2867dd09577 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:58.97873287 +0000 UTC m=+17.228692134 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret") pod "global-pull-secret-syncer-dwhwz" (UID: "c8c7dd60-3fc1-45b5-baae-d2867dd09577") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:50.978905 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:50.978790 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28 podName:071ff18b-ef9e-46ef-a714-e167ad2b30d7 nodeName:}" failed. No retries permitted until 2026-07-08 09:06:58.978780564 +0000 UTC m=+17.228739829 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxx28" (UniqueName: "kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28") pod "network-check-target-g67tm" (UID: "071ff18b-ef9e-46ef-a714-e167ad2b30d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:51.302378 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:51.302296 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:51.302378 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:51.302356 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:51.302378 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:51.302307 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:51.302641 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:51.302454 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:06:51.302641 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:51.302568 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:06:51.302744 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:51.302644 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:06:53.302904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:53.302373 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:53.302904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:53.302407 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:53.302904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:53.302381 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:53.302904 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:53.302498 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:06:53.302904 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:53.302628 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:06:53.302904 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:53.302738 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:06:55.303678 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:55.302970 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:55.303678 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:55.303072 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:55.303678 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:55.303316 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:06:55.303678 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:55.303104 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:55.303678 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:55.303490 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:06:55.303678 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:55.303357 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:06:55.402020 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:55.401978 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal" event={"ID":"802e55c5ab26814202be9eac69956172","Type":"ContainerStarted","Data":"32b41402f672cba6ae14b2e8a77e5cbbc985d98a4c8a079b8ed85a53172bae0d"} Jul 08 09:06:55.403655 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:55.403625 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-297dp" event={"ID":"2d1e824a-40d2-48f9-8ca1-e8f69f686beb","Type":"ContainerStarted","Data":"e49d993e6625ccde681050238a44306b7a673e21a55929e265900ba0bebc24a5"} Jul 08 09:06:55.419528 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:55.419468 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-68.ec2.internal" podStartSLOduration=12.419447752 podStartE2EDuration="12.419447752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:06:55.418762619 +0000 UTC m=+13.668721919" watchObservedRunningTime="2026-07-08 09:06:55.419447752 +0000 UTC m=+13.669407036" Jul 08 09:06:55.436216 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:55.436168 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-297dp" podStartSLOduration=2.069663456 podStartE2EDuration="13.436152432s" totalImagesPullingTime="11.366488976s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:06:55.435684603 +0000 UTC m=+13.685643886" watchObservedRunningTime="2026-07-08 09:06:55.436152432 +0000 UTC m=+13.686111716" Jul 08 09:06:56.407378 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:56.407344 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-65jmn" event={"ID":"61ca3fd1-d587-4a48-a64e-d2eaf482dc92","Type":"ContainerStarted","Data":"47a9cd6e8d544075afd7cf97a78bf15adfd774137ecf92b14045e233e8dd8caf"} Jul 08 09:06:56.409108 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:56.409083 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ht7f6" event={"ID":"048d9fac-8d08-46a7-a4c6-bae6f85821b6","Type":"ContainerStarted","Data":"1b60b96a59940838fdf59f1a05981aca4088de04a393a371c71d34cc3d8eaff5"} Jul 08 09:06:56.410664 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:56.410637 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pbfjd" event={"ID":"ef4d7d33-c0ca-4739-9c59-4b1c1b20a142","Type":"ContainerStarted","Data":"71de7d1ffa450ca5814b72c19746a209fb2759c47e0ad68491649d33a9c89b41"} Jul 08 09:06:56.412254 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:56.412233 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" event={"ID":"e8925c20d0f787a2bb875b826549cafa","Type":"ContainerStarted","Data":"c7111fef7af553e04fb3fb5c42421271b91b4d312bae6774465ac8469f6b15dd"} Jul 08 09:06:56.413946 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:56.413921 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" event={"ID":"d3db75a5-55ba-4be6-99f9-eb9712ff28c7","Type":"ContainerStarted","Data":"db7390b6c0abd6b364001b539a3a09f8370ce1f6c703dc0cff00e1fc3d516548"} Jul 08 09:06:56.415431 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:56.415406 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" event={"ID":"a6c58c8c-52bc-4924-92f2-13063e6bc4de","Type":"ContainerStarted","Data":"bb98dc375899ceec939fd1b5fa7533a02ee9138905ad177e13e08fba330972da"} Jul 08 09:06:56.424486 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:56.424412 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-65jmn" podStartSLOduration=3.139473551 podStartE2EDuration="14.42439919s" totalImagesPullingTime="11.284925639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:06:56.423994476 +0000 UTC m=+14.673953791" watchObservedRunningTime="2026-07-08 09:06:56.42439919 +0000 UTC m=+14.674358522" Jul 08 09:06:56.441822 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:56.441771 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pbfjd" podStartSLOduration=3.085963929 podStartE2EDuration="14.441737951s" totalImagesPullingTime="11.355774022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:06:56.441363539 +0000 UTC m=+14.691322822" watchObservedRunningTime="2026-07-08 09:06:56.441737951 +0000 UTC m=+14.691697241" Jul 08 09:06:56.462359 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:56.462310 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ht7f6" podStartSLOduration=3.175661033 podStartE2EDuration="14.462294614s" totalImagesPullingTime="11.286633581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:06:56.462035702 +0000 UTC m=+14.711994986" watchObservedRunningTime="2026-07-08 09:06:56.462294614 +0000 UTC m=+14.712253899" Jul 08 09:06:57.303044 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:57.303007 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:57.303250 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:57.303050 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:57.303250 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:57.303055 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:57.303250 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:57.303152 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:06:57.303383 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:57.303255 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:06:57.303383 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:57.303345 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:06:57.663283 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:57.663202 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:06:57.663882 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:57.663858 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:06:58.946248 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:58.946209 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:58.946691 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:58.946348 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:58.946691 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:58.946413 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs podName:b9eeaebc-02be-4f69-b34d-6f8135fc5374 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:14.946394685 +0000 UTC m=+33.196353946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs") pod "network-metrics-daemon-lsqn9" (UID: "b9eeaebc-02be-4f69-b34d-6f8135fc5374") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:06:59.046813 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:59.046774 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx28\" (UniqueName: \"kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28\") pod \"network-check-target-g67tm\" (UID: \"071ff18b-ef9e-46ef-a714-e167ad2b30d7\") " pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:59.047023 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:59.046842 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:59.047023 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:59.046946 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:06:59.047023 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:59.046964 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:59.047023 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:59.046969 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:06:59.047023 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:59.046982 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxx28 for pod openshift-network-diagnostics/network-check-target-g67tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:59.047023 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:59.047022 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret podName:c8c7dd60-3fc1-45b5-baae-d2867dd09577 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:15.047007613 +0000 UTC m=+33.296966874 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret") pod "global-pull-secret-syncer-dwhwz" (UID: "c8c7dd60-3fc1-45b5-baae-d2867dd09577") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:06:59.047284 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:59.047036 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28 podName:071ff18b-ef9e-46ef-a714-e167ad2b30d7 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:15.047030489 +0000 UTC m=+33.296989749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxx28" (UniqueName: "kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28") pod "network-check-target-g67tm" (UID: "071ff18b-ef9e-46ef-a714-e167ad2b30d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:06:59.302692 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:59.302663 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:06:59.302866 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:59.302667 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:06:59.302866 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:59.302789 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:06:59.302866 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:59.302667 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:06:59.303014 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:59.302888 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:06:59.303014 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:06:59.302947 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:06:59.423905 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:59.423872 2572 generic.go:356] "Generic (PLEG): container finished" podID="a6c58c8c-52bc-4924-92f2-13063e6bc4de" containerID="bb98dc375899ceec939fd1b5fa7533a02ee9138905ad177e13e08fba330972da" exitCode=0 Jul 08 09:06:59.424056 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:59.423942 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" event={"ID":"a6c58c8c-52bc-4924-92f2-13063e6bc4de","Type":"ContainerDied","Data":"bb98dc375899ceec939fd1b5fa7533a02ee9138905ad177e13e08fba330972da"} Jul 08 09:06:59.425931 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:59.425774 2572 generic.go:356] "Generic (PLEG): container finished" podID="e8925c20d0f787a2bb875b826549cafa" containerID="c7111fef7af553e04fb3fb5c42421271b91b4d312bae6774465ac8469f6b15dd" exitCode=0 Jul 08 09:06:59.425978 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:59.425837 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" event={"ID":"e8925c20d0f787a2bb875b826549cafa","Type":"ContainerDied","Data":"c7111fef7af553e04fb3fb5c42421271b91b4d312bae6774465ac8469f6b15dd"} Jul 08 09:06:59.426011 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:06:59.425984 2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 09:07:01.302091 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:01.302048 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:01.302091 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:01.302088 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:01.302548 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:01.302088 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:01.302548 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:01.302167 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:07:01.302548 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:01.302286 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:07:01.302548 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:01.302359 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:07:03.302981 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:03.302945 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:03.302981 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:03.302967 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:03.303469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:03.303078 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:03.303469 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:03.303089 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:07:03.303469 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:03.303166 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:07:03.303469 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:03.303278 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:07:04.253011 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.252813 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 09:07:04.258064 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.257987 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T09:07:04.25300611Z","UUID":"057e85e1-b6d7-42ac-9f44-b1896366b90e","Handler":null,"Name":"","Endpoint":""} Jul 08 09:07:04.259402 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.259387 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 09:07:04.259491 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.259422 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 09:07:04.439436 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.439396 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" event={"ID":"d3db75a5-55ba-4be6-99f9-eb9712ff28c7","Type":"ContainerStarted","Data":"e8422498aa6a0a307ceceb41d194db44a19b5dce9d926e5efc1f7acd0b6dc05a"} Jul 08 09:07:04.441146 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.441120 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9vq6j" event={"ID":"3b97acfe-ed2a-415d-9223-746ddcdb5115","Type":"ContainerStarted","Data":"771417a3859f7797277c60450592d49ebaa0b993f7edc4fee4ead1f7d21fdb35"} Jul 08 09:07:04.444158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.444129 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" event={"ID":"2457b473-47d8-4489-9ef2-832c2aaaedd1","Type":"ContainerStarted","Data":"b437946d3f4c5ac2e14e86c9e639faf2f5c44d8247a8bb3a9776db9c08417a7d"} Jul 08 09:07:04.444250 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.444207 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" event={"ID":"2457b473-47d8-4489-9ef2-832c2aaaedd1","Type":"ContainerStarted","Data":"2dada0a348ba67c92edb25b32ba2a9459420355260040ee61a77fc0f1445763e"} Jul 08 09:07:04.444320 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.444259 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" event={"ID":"2457b473-47d8-4489-9ef2-832c2aaaedd1","Type":"ContainerStarted","Data":"a5939cde38d80c85fa989f7c5d0c8ac867dc0c97cee8c857ac375dac85f5cb22"} Jul 08 09:07:04.446170 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.446150 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" event={"ID":"e8925c20d0f787a2bb875b826549cafa","Type":"ContainerStarted","Data":"c5fc1cfc000c04ae88aaf007dacd5a7e77d1799956f498f798bd6588ff4a8331"} Jul 08 09:07:04.467566 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.467508 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9vq6j" podStartSLOduration=2.264269502 podStartE2EDuration="22.467470404s" totalImagesPullingTime="20.203200902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:04.467213203 +0000 UTC m=+22.717172497" watchObservedRunningTime="2026-07-08 09:07:04.467470404 +0000 UTC m=+22.717429679" Jul 08 09:07:04.496578 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:04.496532 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-68.ec2.internal" podStartSLOduration=21.496515514 podStartE2EDuration="21.496515514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:04.495586503 +0000 UTC m=+22.745545787" watchObservedRunningTime="2026-07-08 09:07:04.496515514 +0000 UTC m=+22.746474796" Jul 08 09:07:05.302530 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:05.302507 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:05.302593 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:05.302538 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:05.302593 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:05.302541 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:05.302672 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:05.302625 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:07:05.302707 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:05.302669 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:07:05.302768 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:05.302737 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:07:05.450966 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:05.450934 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" event={"ID":"2457b473-47d8-4489-9ef2-832c2aaaedd1","Type":"ContainerStarted","Data":"11ff171d4258fd9c6bedddfb99febc429735bd82bd21099e50f0bc477f704367"} Jul 08 09:07:05.451382 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:05.451007 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" event={"ID":"2457b473-47d8-4489-9ef2-832c2aaaedd1","Type":"ContainerStarted","Data":"53dcf71a46603f87ef0e186e7637be85f1e872f751a8baf7cab80fa2aeeacc00"} Jul 08 09:07:05.451382 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:05.451037 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" event={"ID":"2457b473-47d8-4489-9ef2-832c2aaaedd1","Type":"ContainerStarted","Data":"813882e692b994da588a651f1e574be1dea669cfb9b4a7c9b17d135fa2ae0835"} Jul 08 09:07:05.454099 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:05.454075 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bkll2" event={"ID":"9b3d1f87-66be-4e09-99c1-c1fa7b7ba552","Type":"ContainerStarted","Data":"7db85346bb946f7fab82464dd65a77460c5c57c4aa0223378ee07e93c8c2f3b6"} Jul 08 09:07:05.471282 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:05.471237 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bkll2" podStartSLOduration=23.471220917 podStartE2EDuration="23.471220917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-08 09:07:05.470854691 +0000 UTC m=+23.720813973" watchObservedRunningTime="2026-07-08 09:07:05.471220917 +0000 UTC m=+23.721180199" Jul 08 09:07:06.458836 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:06.458795 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" event={"ID":"d3db75a5-55ba-4be6-99f9-eb9712ff28c7","Type":"ContainerStarted","Data":"3a6a493e6e07eb5c390f6ede3afdc23969182447417d1ba41771d3a4ab8f9be3"} Jul 08 09:07:06.480299 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:06.480250 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qgrnt" podStartSLOduration=3.9404452340000002 podStartE2EDuration="24.480232186s" totalImagesPullingTime="20.539786952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:06.479918467 +0000 UTC m=+24.729877750" watchObservedRunningTime="2026-07-08 09:07:06.480232186 +0000 UTC m=+24.730191471" Jul 08 09:07:07.303122 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:07.302936 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:07.303311 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:07.302935 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:07.303311 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:07.303222 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:07:07.303311 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:07.302935 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:07.303311 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:07.303273 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:07:07.303530 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:07.303360 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:07:07.465487 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:07.465446 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" event={"ID":"2457b473-47d8-4489-9ef2-832c2aaaedd1","Type":"ContainerStarted","Data":"e0629ea3f02f465407c2000ccc92a59b2a2a5b690de65307c595bef7c64d8d17"} Jul 08 09:07:09.302712 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:09.302626 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:09.303390 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:09.302626 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:09.303390 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:09.302627 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:09.303390 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:09.302773 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:07:09.303390 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:09.302835 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:07:09.303390 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:09.302929 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:07:10.474094 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:10.473909 2572 generic.go:356] "Generic (PLEG): container finished" podID="a6c58c8c-52bc-4924-92f2-13063e6bc4de" containerID="ff021a85d461ee4680228298e43380bca79a1afaa973af4f13942bad25cf68b4" exitCode=0 Jul 08 09:07:10.474704 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:10.473981 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" event={"ID":"a6c58c8c-52bc-4924-92f2-13063e6bc4de","Type":"ContainerDied","Data":"ff021a85d461ee4680228298e43380bca79a1afaa973af4f13942bad25cf68b4"} Jul 08 09:07:10.478017 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:10.477996 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" event={"ID":"2457b473-47d8-4489-9ef2-832c2aaaedd1","Type":"ContainerStarted","Data":"ada7eb20969257a84375f5e1df46d0fa161ce1752e112042521be89888394a62"} Jul 08 09:07:10.478303 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:10.478286 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:07:10.493769 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:10.493731 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:07:10.536861 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:10.536814 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" podStartSLOduration=8.290769411 podStartE2EDuration="28.536799431s" totalImagesPullingTime="20.24603002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:10.536281867 +0000 UTC m=+28.786241179" watchObservedRunningTime="2026-07-08 09:07:10.536799431 +0000 UTC m=+28.786758714" Jul 08 09:07:11.302385 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.302345 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:11.302558 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.302452 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:11.302558 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:11.302458 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:07:11.302558 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.302490 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:11.302651 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:11.302608 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:07:11.302724 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:11.302702 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:07:11.483220 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.483187 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" event={"ID":"a6c58c8c-52bc-4924-92f2-13063e6bc4de","Type":"ContainerStarted","Data":"93b28d35e7eff3730e723c0b1eb3aa5c5284b5797d13ee63fa3c57b17e16d981"} Jul 08 09:07:11.487305 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.485628 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:07:11.487305 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.485743 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:07:11.500319 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.500293 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:07:11.687699 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.687618 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lsqn9"] Jul 08 09:07:11.687842 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.687740 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:11.687882 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:11.687858 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:07:11.690946 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.690925 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dwhwz"] Jul 08 09:07:11.691066 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.691011 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:11.691109 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:11.691090 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:07:11.700461 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.700434 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g67tm"] Jul 08 09:07:11.700553 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.700540 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:11.700657 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:11.700637 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:07:11.781691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.781662 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:07:11.781872 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.781787 2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 09:07:11.782217 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:11.782196 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pbfjd" Jul 08 09:07:12.487830 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:12.487791 2572 generic.go:356] "Generic (PLEG): container finished" podID="a6c58c8c-52bc-4924-92f2-13063e6bc4de" containerID="93b28d35e7eff3730e723c0b1eb3aa5c5284b5797d13ee63fa3c57b17e16d981" exitCode=0 Jul 08 09:07:12.488216 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:12.487873 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" event={"ID":"a6c58c8c-52bc-4924-92f2-13063e6bc4de","Type":"ContainerDied","Data":"93b28d35e7eff3730e723c0b1eb3aa5c5284b5797d13ee63fa3c57b17e16d981"} Jul 08 09:07:13.302742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:13.302518 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:13.302896 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:13.302568 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:13.302896 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:13.302820 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:07:13.302896 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:13.302568 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:13.302896 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:13.302876 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:07:13.303041 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:13.302926 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:07:14.495134 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:14.495105 2572 generic.go:356] "Generic (PLEG): container finished" podID="a6c58c8c-52bc-4924-92f2-13063e6bc4de" containerID="9b105e30f7c862d6b19dacda81b02617cfc4972f06b64b71ae9db79cafbfa85f" exitCode=0 Jul 08 09:07:14.495656 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:14.495156 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" event={"ID":"a6c58c8c-52bc-4924-92f2-13063e6bc4de","Type":"ContainerDied","Data":"9b105e30f7c862d6b19dacda81b02617cfc4972f06b64b71ae9db79cafbfa85f"} Jul 08 09:07:14.968928 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:14.968846 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:14.969052 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:14.968965 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:14.969052 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:14.969014 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs podName:b9eeaebc-02be-4f69-b34d-6f8135fc5374 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:46.969001384 +0000 UTC m=+65.218960645 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs") pod "network-metrics-daemon-lsqn9" (UID: "b9eeaebc-02be-4f69-b34d-6f8135fc5374") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:15.069169 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:15.069136 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:15.069346 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:15.069196 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx28\" (UniqueName: \"kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28\") pod \"network-check-target-g67tm\" (UID: \"071ff18b-ef9e-46ef-a714-e167ad2b30d7\") " pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:15.069346 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:15.069289 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:15.069448 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:15.069358 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret podName:c8c7dd60-3fc1-45b5-baae-d2867dd09577 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:47.069341071 +0000 UTC m=+65.319300332 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret") pod "global-pull-secret-syncer-dwhwz" (UID: "c8c7dd60-3fc1-45b5-baae-d2867dd09577") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:15.069448 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:15.069298 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:07:15.069448 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:15.069390 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:07:15.069448 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:15.069402 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxx28 for pod openshift-network-diagnostics/network-check-target-g67tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:15.069448 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:15.069447 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28 podName:071ff18b-ef9e-46ef-a714-e167ad2b30d7 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:47.069434268 +0000 UTC m=+65.319393532 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxx28" (UniqueName: "kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28") pod "network-check-target-g67tm" (UID: "071ff18b-ef9e-46ef-a714-e167ad2b30d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:15.302215 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:15.302123 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:15.302380 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:15.302124 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:15.302380 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:15.302261 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsqn9" podUID="b9eeaebc-02be-4f69-b34d-6f8135fc5374" Jul 08 09:07:15.302380 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:15.302124 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:15.302380 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:15.302314 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwhwz" podUID="c8c7dd60-3fc1-45b5-baae-d2867dd09577" Jul 08 09:07:15.302380 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:15.302379 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g67tm" podUID="071ff18b-ef9e-46ef-a714-e167ad2b30d7" Jul 08 09:07:16.548727 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.548654 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-68.ec2.internal" event="NodeReady" Jul 08 09:07:16.549212 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.548798 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 09:07:16.587321 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.587290 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-676bb5bbc6-xjfxl"] Jul 08 09:07:16.589297 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.589269 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.592047 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.592003 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 09:07:16.592158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.592091 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 09:07:16.592158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.592009 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 09:07:16.592461 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.592441 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zkvd5\"" Jul 08 09:07:16.598081 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.597791 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 09:07:16.605916 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.605869 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zg4rz"] Jul 08 09:07:16.607946 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.607924 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8g5pt"] Jul 08 09:07:16.608069 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.608052 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.609960 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.609936 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8g5pt" Jul 08 09:07:16.610915 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.610893 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 09:07:16.611022 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.610983 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 09:07:16.611088 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.610899 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qglp2\"" Jul 08 09:07:16.611603 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.611588 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676bb5bbc6-xjfxl"] Jul 08 09:07:16.613631 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.613609 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 09:07:16.613699 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.613613 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 09:07:16.613699 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.613661 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8jcgn\"" Jul 08 09:07:16.613797 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.613617 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 09:07:16.620872 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.620850 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8g5pt"] Jul 08 09:07:16.625902 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.625884 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zg4rz"] Jul 08 09:07:16.681540 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.681509 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.681540 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.681542 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d8c644b8-b197-4382-8e5f-806cdc3323f5-ca-trust-extracted\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.681739 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.681562 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-bound-sa-token\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.681739 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.681622 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-certificates\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.681739 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.681666 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-installation-pull-secrets\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.681739 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.681688 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8dnt\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-kube-api-access-q8dnt\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.681739 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.681724 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-image-registry-private-configuration\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.681916 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.681773 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-trusted-ca\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.782310 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782277 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-trusted-ca\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.782484 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782329 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.782484 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782350 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d8c644b8-b197-4382-8e5f-806cdc3323f5-ca-trust-extracted\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.782484 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782378 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-bound-sa-token\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.782484 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782405 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.782484 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782444 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-certificates\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.782484 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782467 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/152b5ffb-392f-4f27-9379-ca71ab6070b6-cert\") pod \"ingress-canary-8g5pt\" (UID: \"152b5ffb-392f-4f27-9379-ca71ab6070b6\") " pod="openshift-ingress-canary/ingress-canary-8g5pt" Jul 08 09:07:16.782484 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:16.782463 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:07:16.782829 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:16.782501 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676bb5bbc6-xjfxl: secret "image-registry-tls" not found Jul 08 09:07:16.782829 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:16.782564 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls podName:d8c644b8-b197-4382-8e5f-806cdc3323f5 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:17.282545617 +0000 UTC m=+35.532504882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls") pod "image-registry-676bb5bbc6-xjfxl" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5") : secret "image-registry-tls" not found Jul 08 09:07:16.782829 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782598 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxkjl\" (UniqueName: \"kubernetes.io/projected/152b5ffb-392f-4f27-9379-ca71ab6070b6-kube-api-access-vxkjl\") pod \"ingress-canary-8g5pt\" (UID: \"152b5ffb-392f-4f27-9379-ca71ab6070b6\") " pod="openshift-ingress-canary/ingress-canary-8g5pt" Jul 08 09:07:16.782829 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782634 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-installation-pull-secrets\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.782829 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782658 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8dnt\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-kube-api-access-q8dnt\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.782829 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782683 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xbp7\" (UniqueName: \"kubernetes.io/projected/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-kube-api-access-5xbp7\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.782829 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782737 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-image-registry-private-configuration\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.782829 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782789 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-config-volume\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.782829 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782811 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-tmp-dir\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.783212 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.782847 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d8c644b8-b197-4382-8e5f-806cdc3323f5-ca-trust-extracted\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.783431 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.783402 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-certificates\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.783538 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.783454 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-trusted-ca\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.787350 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.787326 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-image-registry-private-configuration\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.787350 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.787337 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-installation-pull-secrets\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.791986 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.791962 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8dnt\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-kube-api-access-q8dnt\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.792465 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.792449 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-bound-sa-token\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:16.883812 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.883720 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.883812 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.883785 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/152b5ffb-392f-4f27-9379-ca71ab6070b6-cert\") pod \"ingress-canary-8g5pt\" (UID: \"152b5ffb-392f-4f27-9379-ca71ab6070b6\") " pod="openshift-ingress-canary/ingress-canary-8g5pt" Jul 08 09:07:16.884019 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:16.883892 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:16.884019 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.883916 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxkjl\" (UniqueName: \"kubernetes.io/projected/152b5ffb-392f-4f27-9379-ca71ab6070b6-kube-api-access-vxkjl\") pod \"ingress-canary-8g5pt\" (UID: \"152b5ffb-392f-4f27-9379-ca71ab6070b6\") " pod="openshift-ingress-canary/ingress-canary-8g5pt" Jul 08 09:07:16.884019 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:16.883971 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls podName:be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:17.383954285 +0000 UTC m=+35.633913545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls") pod "dns-default-zg4rz" (UID: "be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8") : secret "dns-default-metrics-tls" not found Jul 08 09:07:16.884019 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.883988 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xbp7\" (UniqueName: \"kubernetes.io/projected/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-kube-api-access-5xbp7\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.884198 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.884031 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-config-volume\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.884198 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.884053 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-tmp-dir\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.884391 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.884372 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-tmp-dir\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.884612 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.884596 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-config-volume\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.886283 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.886263 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/152b5ffb-392f-4f27-9379-ca71ab6070b6-cert\") pod \"ingress-canary-8g5pt\" (UID: \"152b5ffb-392f-4f27-9379-ca71ab6070b6\") " pod="openshift-ingress-canary/ingress-canary-8g5pt" Jul 08 09:07:16.894274 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.894241 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xbp7\" (UniqueName: \"kubernetes.io/projected/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-kube-api-access-5xbp7\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:16.894385 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.894358 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxkjl\" (UniqueName: \"kubernetes.io/projected/152b5ffb-392f-4f27-9379-ca71ab6070b6-kube-api-access-vxkjl\") pod \"ingress-canary-8g5pt\" (UID: \"152b5ffb-392f-4f27-9379-ca71ab6070b6\") " pod="openshift-ingress-canary/ingress-canary-8g5pt" Jul 08 09:07:16.927369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:16.927337 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8g5pt" Jul 08 09:07:17.081237 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.081034 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8g5pt"] Jul 08 09:07:17.085479 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:07:17.085444 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod152b5ffb_392f_4f27_9379_ca71ab6070b6.slice/crio-ef286834d6dd254cd0c23c7e5270284d55737a6bd385d1a540f5ddd1405d2802 WatchSource:0}: Error finding container ef286834d6dd254cd0c23c7e5270284d55737a6bd385d1a540f5ddd1405d2802: Status 404 returned error can't find the container with id ef286834d6dd254cd0c23c7e5270284d55737a6bd385d1a540f5ddd1405d2802 Jul 08 09:07:17.289088 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.289044 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:17.289234 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:17.289205 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:07:17.289234 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:17.289221 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676bb5bbc6-xjfxl: secret "image-registry-tls" not found Jul 08 09:07:17.289310 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:17.289272 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls podName:d8c644b8-b197-4382-8e5f-806cdc3323f5 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:18.289257543 +0000 UTC m=+36.539216806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls") pod "image-registry-676bb5bbc6-xjfxl" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5") : secret "image-registry-tls" not found Jul 08 09:07:17.302503 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.302477 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:17.302664 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.302504 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:17.302664 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.302477 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:17.307278 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.307252 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 09:07:17.307431 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.307370 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 09:07:17.307656 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.307597 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6bnkm\"" Jul 08 09:07:17.307793 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.307706 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gctfj\"" Jul 08 09:07:17.307857 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.307797 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 09:07:17.308523 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.308505 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 09:07:17.389766 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.389715 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:17.389932 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:17.389874 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:17.389969 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:17.389940 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls podName:be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:18.389920661 +0000 UTC m=+36.639879925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls") pod "dns-default-zg4rz" (UID: "be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8") : secret "dns-default-metrics-tls" not found Jul 08 09:07:17.505163 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:17.505099 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8g5pt" event={"ID":"152b5ffb-392f-4f27-9379-ca71ab6070b6","Type":"ContainerStarted","Data":"ef286834d6dd254cd0c23c7e5270284d55737a6bd385d1a540f5ddd1405d2802"} Jul 08 09:07:18.299180 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:18.299108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:18.299715 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:18.299268 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:07:18.299715 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:18.299286 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676bb5bbc6-xjfxl: secret "image-registry-tls" not found Jul 08 09:07:18.299715 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:18.299357 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls podName:d8c644b8-b197-4382-8e5f-806cdc3323f5 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:20.299331766 +0000 UTC m=+38.549291029 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls") pod "image-registry-676bb5bbc6-xjfxl" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5") : secret "image-registry-tls" not found Jul 08 09:07:18.400444 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:18.400411 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:18.400599 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:18.400573 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:18.400648 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:18.400643 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls podName:be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:20.400626742 +0000 UTC m=+38.650586004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls") pod "dns-default-zg4rz" (UID: "be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8") : secret "dns-default-metrics-tls" not found Jul 08 09:07:20.318039 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:20.318008 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:20.318482 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:20.318178 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:07:20.318482 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:20.318195 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676bb5bbc6-xjfxl: secret "image-registry-tls" not found Jul 08 09:07:20.318482 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:20.318250 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls podName:d8c644b8-b197-4382-8e5f-806cdc3323f5 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:24.318233036 +0000 UTC m=+42.568192307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls") pod "image-registry-676bb5bbc6-xjfxl" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5") : secret "image-registry-tls" not found Jul 08 09:07:20.419347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:20.419304 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:20.419541 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:20.419481 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:20.419607 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:20.419548 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls podName:be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:24.419529151 +0000 UTC m=+42.669488412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls") pod "dns-default-zg4rz" (UID: "be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8") : secret "dns-default-metrics-tls" not found Jul 08 09:07:22.519316 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:22.519092 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8g5pt" event={"ID":"152b5ffb-392f-4f27-9379-ca71ab6070b6","Type":"ContainerStarted","Data":"873b4aaa48c9c0a6e4dd2fd4b598904f203169f151aca4ea4c0b4cc72975f9f1"} Jul 08 09:07:22.521724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:22.521698 2572 generic.go:356] "Generic (PLEG): container finished" podID="a6c58c8c-52bc-4924-92f2-13063e6bc4de" containerID="29e6a60837246fbd44cd283f07240f7bb8ad2c46f47b63c922ced20b4bfb8c5f" exitCode=0 Jul 08 09:07:22.521854 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:22.521735 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" event={"ID":"a6c58c8c-52bc-4924-92f2-13063e6bc4de","Type":"ContainerDied","Data":"29e6a60837246fbd44cd283f07240f7bb8ad2c46f47b63c922ced20b4bfb8c5f"} Jul 08 09:07:22.569608 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:22.569552 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8g5pt" podStartSLOduration=1.534537217 podStartE2EDuration="6.569535495s" totalImagesPullingTime="5.034998278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:22.538431363 +0000 UTC m=+40.788390646" watchObservedRunningTime="2026-07-08 09:07:22.569535495 +0000 UTC m=+40.819494777" Jul 08 09:07:23.526857 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:23.526825 2572 generic.go:356] "Generic (PLEG): container finished" podID="a6c58c8c-52bc-4924-92f2-13063e6bc4de" containerID="d2bf370ff4ccab7d23e4df2803301f8141390aeb9d1a742060f945352a7e0ed8" exitCode=0 Jul 08 09:07:23.527222 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:23.526901 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" event={"ID":"a6c58c8c-52bc-4924-92f2-13063e6bc4de","Type":"ContainerDied","Data":"d2bf370ff4ccab7d23e4df2803301f8141390aeb9d1a742060f945352a7e0ed8"} Jul 08 09:07:24.350800 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:24.350737 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:24.350975 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:24.350892 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:07:24.350975 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:24.350911 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676bb5bbc6-xjfxl: secret "image-registry-tls" not found Jul 08 09:07:24.350975 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:24.350965 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls podName:d8c644b8-b197-4382-8e5f-806cdc3323f5 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:32.350951294 +0000 UTC m=+50.600910554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls") pod "image-registry-676bb5bbc6-xjfxl" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5") : secret "image-registry-tls" not found Jul 08 09:07:24.451284 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:24.451248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:24.451435 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:24.451366 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:24.451435 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:24.451416 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls podName:be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:32.451402691 +0000 UTC m=+50.701361958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls") pod "dns-default-zg4rz" (UID: "be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8") : secret "dns-default-metrics-tls" not found Jul 08 09:07:24.535812 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:24.535777 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" event={"ID":"a6c58c8c-52bc-4924-92f2-13063e6bc4de","Type":"ContainerStarted","Data":"221b8684d62e7ca4ab0a26290b2b6c2f115f77a704f7600684e2295bf6a48016"} Jul 08 09:07:24.566992 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:24.566363 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mfpzs" podStartSLOduration=11.15922446 podStartE2EDuration="42.566346809s" totalImagesPullingTime="31.407122349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:24.564500494 +0000 UTC m=+42.814459776" watchObservedRunningTime="2026-07-08 09:07:24.566346809 +0000 UTC m=+42.816306069" Jul 08 09:07:32.410248 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:32.410208 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:32.410746 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:32.410369 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:07:32.410746 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:32.410392 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676bb5bbc6-xjfxl: secret "image-registry-tls" not found Jul 08 09:07:32.410746 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:32.410448 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls podName:d8c644b8-b197-4382-8e5f-806cdc3323f5 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:48.410432295 +0000 UTC m=+66.660391559 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls") pod "image-registry-676bb5bbc6-xjfxl" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5") : secret "image-registry-tls" not found Jul 08 09:07:32.510726 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:32.510687 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:32.510896 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:32.510874 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:32.510965 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:32.510954 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls podName:be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:48.510933309 +0000 UTC m=+66.760892587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls") pod "dns-default-zg4rz" (UID: "be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8") : secret "dns-default-metrics-tls" not found Jul 08 09:07:43.500687 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:43.500655 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pw42g" Jul 08 09:07:47.019170 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.019134 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:07:47.023362 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.023343 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 09:07:47.029973 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:47.029955 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 09:07:47.030066 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:47.030024 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs podName:b9eeaebc-02be-4f69-b34d-6f8135fc5374 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:51.03000903 +0000 UTC m=+129.279968291 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs") pod "network-metrics-daemon-lsqn9" (UID: "b9eeaebc-02be-4f69-b34d-6f8135fc5374") : secret "metrics-daemon-secret" not found Jul 08 09:07:47.119895 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.119850 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx28\" (UniqueName: \"kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28\") pod \"network-check-target-g67tm\" (UID: \"071ff18b-ef9e-46ef-a714-e167ad2b30d7\") " pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:47.120021 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.119912 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:47.122723 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.122700 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 09:07:47.122836 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.122812 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 09:07:47.133871 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.133847 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 09:07:47.134797 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.134777 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8c7dd60-3fc1-45b5-baae-d2867dd09577-original-pull-secret\") pod \"global-pull-secret-syncer-dwhwz\" (UID: \"c8c7dd60-3fc1-45b5-baae-d2867dd09577\") " pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:47.143538 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.143517 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxx28\" (UniqueName: \"kubernetes.io/projected/071ff18b-ef9e-46ef-a714-e167ad2b30d7-kube-api-access-sxx28\") pod \"network-check-target-g67tm\" (UID: \"071ff18b-ef9e-46ef-a714-e167ad2b30d7\") " pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:47.317815 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.317717 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gctfj\"" Jul 08 09:07:47.321514 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.321496 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwhwz" Jul 08 09:07:47.323899 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.323878 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:47.489518 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.489487 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g67tm"] Jul 08 09:07:47.493663 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:07:47.493637 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071ff18b_ef9e_46ef_a714_e167ad2b30d7.slice/crio-b9d54067a5f06c21b937625395908a24a0246aa7e2a3e0e61fc3fe6195cdec74 WatchSource:0}: Error finding container b9d54067a5f06c21b937625395908a24a0246aa7e2a3e0e61fc3fe6195cdec74: Status 404 returned error can't find the container with id b9d54067a5f06c21b937625395908a24a0246aa7e2a3e0e61fc3fe6195cdec74 Jul 08 09:07:47.503107 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.503084 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dwhwz"] Jul 08 09:07:47.506057 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:07:47.506033 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c7dd60_3fc1_45b5_baae_d2867dd09577.slice/crio-f92203cc3c51db75f067dbdbc258e21fc36f3af90b4eed32255baa0ea4da5efe WatchSource:0}: Error finding container f92203cc3c51db75f067dbdbc258e21fc36f3af90b4eed32255baa0ea4da5efe: Status 404 returned error can't find the container with id f92203cc3c51db75f067dbdbc258e21fc36f3af90b4eed32255baa0ea4da5efe Jul 08 09:07:47.586714 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.586621 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g67tm" event={"ID":"071ff18b-ef9e-46ef-a714-e167ad2b30d7","Type":"ContainerStarted","Data":"b9d54067a5f06c21b937625395908a24a0246aa7e2a3e0e61fc3fe6195cdec74"} Jul 08 09:07:47.587683 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:47.587658 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dwhwz" event={"ID":"c8c7dd60-3fc1-45b5-baae-d2867dd09577","Type":"ContainerStarted","Data":"f92203cc3c51db75f067dbdbc258e21fc36f3af90b4eed32255baa0ea4da5efe"} Jul 08 09:07:48.431131 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:48.431088 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:07:48.431690 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:48.431247 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:07:48.431690 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:48.431263 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676bb5bbc6-xjfxl: secret "image-registry-tls" not found Jul 08 09:07:48.431690 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:48.431317 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls podName:d8c644b8-b197-4382-8e5f-806cdc3323f5 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:20.431303097 +0000 UTC m=+98.681262361 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls") pod "image-registry-676bb5bbc6-xjfxl" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5") : secret "image-registry-tls" not found Jul 08 09:07:48.532344 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:48.532298 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:07:48.532541 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:48.532476 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:48.532605 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:07:48.532560 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls podName:be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:20.532542095 +0000 UTC m=+98.782501359 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls") pod "dns-default-zg4rz" (UID: "be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8") : secret "dns-default-metrics-tls" not found Jul 08 09:07:51.600123 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:51.600086 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g67tm" event={"ID":"071ff18b-ef9e-46ef-a714-e167ad2b30d7","Type":"ContainerStarted","Data":"e15a0a5366d5442043e1daa5e12c9151d43f32b88c470bef70574e7389fe36c9"} Jul 08 09:07:51.600579 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:51.600305 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:07:53.605523 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:53.605483 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dwhwz" event={"ID":"c8c7dd60-3fc1-45b5-baae-d2867dd09577","Type":"ContainerStarted","Data":"f9f4d25b41e5f4d1ab75acb8f5af65938dc0d0c878e0dbba4da8fe3eeb7f86d9"} Jul 08 09:07:53.622739 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:53.622685 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g67tm" podStartSLOduration=67.930822848 podStartE2EDuration="1m11.622670838s" totalImagesPullingTime="3.69184799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:51.618884815 +0000 UTC m=+69.868844098" watchObservedRunningTime="2026-07-08 09:07:53.622670838 +0000 UTC m=+71.872630129" Jul 08 09:07:53.622910 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:07:53.622784 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dwhwz" podStartSLOduration=66.171636168 podStartE2EDuration="1m11.622777055s" totalImagesPullingTime="5.451140887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:53.621782614 +0000 UTC m=+71.871741898" watchObservedRunningTime="2026-07-08 09:07:53.622777055 +0000 UTC m=+71.872736335" Jul 08 09:08:10.742388 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.742268 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-db9df69b9-6wcdw"] Jul 08 09:08:10.744764 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.744737 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.748718 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.748697 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 09:08:10.748846 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.748697 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 08 09:08:10.748959 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.748947 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gbk4j\"" Jul 08 09:08:10.749859 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.749835 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 09:08:10.749950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.749913 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 08 09:08:10.749950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.749921 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 08 09:08:10.749950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.749911 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 08 09:08:10.765578 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.765525 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-db9df69b9-6wcdw"] Jul 08 09:08:10.887275 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.887236 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-default-certificate\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.887275 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.887280 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-stats-auth\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.887477 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.887303 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m74xg\" (UniqueName: \"kubernetes.io/projected/57259205-640d-4f5f-9e76-a607791fbe62-kube-api-access-m74xg\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.887477 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.887411 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.887477 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.887451 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.987932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.987903 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.988097 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.987945 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.988097 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.987966 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-default-certificate\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.988097 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:10.988065 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 09:08:10.988097 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.988089 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-stats-auth\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.988277 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:10.988109 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:11.488091365 +0000 UTC m=+89.738050627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : configmap references non-existent config key: service-ca.crt Jul 08 09:08:10.988277 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:10.988135 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:11.488126711 +0000 UTC m=+89.738085972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : secret "router-metrics-certs-default" not found Jul 08 09:08:10.988277 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.988167 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m74xg\" (UniqueName: \"kubernetes.io/projected/57259205-640d-4f5f-9e76-a607791fbe62-kube-api-access-m74xg\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.990544 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.990510 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-default-certificate\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:10.990544 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:10.990529 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-stats-auth\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:11.005039 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:11.004968 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m74xg\" (UniqueName: \"kubernetes.io/projected/57259205-640d-4f5f-9e76-a607791fbe62-kube-api-access-m74xg\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:11.491046 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:11.491014 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:11.491222 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:11.491065 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:11.491222 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:11.491191 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 09:08:11.491342 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:11.491221 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:12.491202758 +0000 UTC m=+90.741162022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : configmap references non-existent config key: service-ca.crt Jul 08 09:08:11.491342 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:11.491254 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:12.491237232 +0000 UTC m=+90.741196499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : secret "router-metrics-certs-default" not found Jul 08 09:08:12.498430 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:12.498382 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:12.498430 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:12.498436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:12.498939 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:12.498554 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:14.498540886 +0000 UTC m=+92.748500152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : configmap references non-existent config key: service-ca.crt Jul 08 09:08:12.498939 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:12.498554 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 09:08:12.498939 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:12.498636 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:14.498619313 +0000 UTC m=+92.748578575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : secret "router-metrics-certs-default" not found Jul 08 09:08:14.514032 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:14.513991 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:14.514032 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:14.514043 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:14.514454 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:14.514146 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 09:08:14.514454 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:14.514160 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:18.514145696 +0000 UTC m=+96.764104957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : configmap references non-existent config key: service-ca.crt Jul 08 09:08:14.514454 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:14.514212 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:18.514197273 +0000 UTC m=+96.764156539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : secret "router-metrics-certs-default" not found Jul 08 09:08:17.755640 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.755607 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9ssv9"] Jul 08 09:08:17.758625 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.758605 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.761235 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.761202 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 09:08:17.761342 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.761266 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 09:08:17.762460 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.762444 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 09:08:17.762536 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.762488 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vc9gl\"" Jul 08 09:08:17.762593 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.762566 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 09:08:17.790468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.790439 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9ssv9"] Jul 08 09:08:17.840891 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.840853 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.841082 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.840917 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.841082 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.840963 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/114b9a1c-0321-4b1d-a570-97c10de99789-data-volume\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.841082 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.841006 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rghhb\" (UniqueName: \"kubernetes.io/projected/114b9a1c-0321-4b1d-a570-97c10de99789-kube-api-access-rghhb\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.841240 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.841184 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/114b9a1c-0321-4b1d-a570-97c10de99789-crio-socket\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.941959 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.941915 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.941959 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.941959 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.942157 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.941991 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/114b9a1c-0321-4b1d-a570-97c10de99789-data-volume\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.942157 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.942015 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rghhb\" (UniqueName: \"kubernetes.io/projected/114b9a1c-0321-4b1d-a570-97c10de99789-kube-api-access-rghhb\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.942157 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:17.942068 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 09:08:17.942157 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:17.942102 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 09:08:17.942157 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.942130 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/114b9a1c-0321-4b1d-a570-97c10de99789-crio-socket\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.942305 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:17.942159 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm podName:114b9a1c-0321-4b1d-a570-97c10de99789 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:18.442137334 +0000 UTC m=+96.692096601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9ssv9" (UID: "114b9a1c-0321-4b1d-a570-97c10de99789") : configmap "kube-rbac-proxy" not found Jul 08 09:08:17.942305 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:17.942185 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls podName:114b9a1c-0321-4b1d-a570-97c10de99789 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:18.44217647 +0000 UTC m=+96.692135732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9ssv9" (UID: "114b9a1c-0321-4b1d-a570-97c10de99789") : secret "insights-runtime-extractor-tls" not found Jul 08 09:08:17.942305 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.942189 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/114b9a1c-0321-4b1d-a570-97c10de99789-crio-socket\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.942417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.942371 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/114b9a1c-0321-4b1d-a570-97c10de99789-data-volume\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:17.960186 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:17.960162 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rghhb\" (UniqueName: \"kubernetes.io/projected/114b9a1c-0321-4b1d-a570-97c10de99789-kube-api-access-rghhb\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:18.445308 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:18.445257 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:18.445308 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:18.445306 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:18.445534 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:18.445384 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 09:08:18.445534 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:18.445408 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 09:08:18.445534 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:18.445470 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls podName:114b9a1c-0321-4b1d-a570-97c10de99789 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:19.4454529 +0000 UTC m=+97.695412162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9ssv9" (UID: "114b9a1c-0321-4b1d-a570-97c10de99789") : secret "insights-runtime-extractor-tls" not found Jul 08 09:08:18.445534 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:18.445491 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm podName:114b9a1c-0321-4b1d-a570-97c10de99789 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:19.445477591 +0000 UTC m=+97.695436852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9ssv9" (UID: "114b9a1c-0321-4b1d-a570-97c10de99789") : configmap "kube-rbac-proxy" not found Jul 08 09:08:18.545743 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:18.545704 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:18.545931 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:18.545841 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:18.545931 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:18.545873 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:26.545855672 +0000 UTC m=+104.795814934 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : configmap references non-existent config key: service-ca.crt Jul 08 09:08:18.546006 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:18.545945 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 09:08:18.546006 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:18.545999 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:26.545986102 +0000 UTC m=+104.795945363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : secret "router-metrics-certs-default" not found Jul 08 09:08:19.452835 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:19.452782 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:19.452835 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:19.452845 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:19.453320 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:19.452899 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 09:08:19.453320 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:19.452972 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm podName:114b9a1c-0321-4b1d-a570-97c10de99789 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:21.452953261 +0000 UTC m=+99.702912532 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9ssv9" (UID: "114b9a1c-0321-4b1d-a570-97c10de99789") : configmap "kube-rbac-proxy" not found Jul 08 09:08:19.453320 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:19.452976 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 09:08:19.453320 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:19.453009 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls podName:114b9a1c-0321-4b1d-a570-97c10de99789 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:21.452999927 +0000 UTC m=+99.702959188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9ssv9" (UID: "114b9a1c-0321-4b1d-a570-97c10de99789") : secret "insights-runtime-extractor-tls" not found Jul 08 09:08:20.462146 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:20.462093 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls\") pod \"image-registry-676bb5bbc6-xjfxl\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:08:20.462542 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:20.462237 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:08:20.462542 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:20.462251 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676bb5bbc6-xjfxl: secret "image-registry-tls" not found Jul 08 09:08:20.462542 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:20.462304 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls podName:d8c644b8-b197-4382-8e5f-806cdc3323f5 nodeName:}" failed. No retries permitted until 2026-07-08 09:09:24.462290199 +0000 UTC m=+162.712249464 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls") pod "image-registry-676bb5bbc6-xjfxl" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5") : secret "image-registry-tls" not found Jul 08 09:08:20.563129 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:20.563095 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:08:20.563346 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:20.563267 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:08:20.563412 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:20.563347 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls podName:be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8 nodeName:}" failed. No retries permitted until 2026-07-08 09:09:24.563327893 +0000 UTC m=+162.813287155 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls") pod "dns-default-zg4rz" (UID: "be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8") : secret "dns-default-metrics-tls" not found Jul 08 09:08:20.761164 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:20.761137 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ht7f6_048d9fac-8d08-46a7-a4c6-bae6f85821b6/dns-node-resolver/0.log" Jul 08 09:08:21.366493 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:21.366466 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-65jmn_61ca3fd1-d587-4a48-a64e-d2eaf482dc92/node-ca/0.log" Jul 08 09:08:21.471348 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:21.471312 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:21.471348 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:21.471350 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:21.471724 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:21.471445 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 09:08:21.471724 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:21.471497 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 09:08:21.471724 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:21.471509 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm podName:114b9a1c-0321-4b1d-a570-97c10de99789 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:25.471492542 +0000 UTC m=+103.721451806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9ssv9" (UID: "114b9a1c-0321-4b1d-a570-97c10de99789") : configmap "kube-rbac-proxy" not found Jul 08 09:08:21.471724 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:21.471537 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls podName:114b9a1c-0321-4b1d-a570-97c10de99789 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:25.471526205 +0000 UTC m=+103.721485465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9ssv9" (UID: "114b9a1c-0321-4b1d-a570-97c10de99789") : secret "insights-runtime-extractor-tls" not found Jul 08 09:08:22.359262 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:22.359201 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8g5pt_152b5ffb-392f-4f27-9379-ca71ab6070b6/serve-healthcheck-canary/0.log" Jul 08 09:08:22.605074 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:22.605049 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g67tm" Jul 08 09:08:25.507014 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:25.506976 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:25.507462 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:25.507028 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:25.507462 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:25.507099 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 09:08:25.507462 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:25.507164 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 09:08:25.507462 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:25.507172 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm podName:114b9a1c-0321-4b1d-a570-97c10de99789 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:33.507154339 +0000 UTC m=+111.757113600 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9ssv9" (UID: "114b9a1c-0321-4b1d-a570-97c10de99789") : configmap "kube-rbac-proxy" not found Jul 08 09:08:25.507462 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:25.507206 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls podName:114b9a1c-0321-4b1d-a570-97c10de99789 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:33.507195102 +0000 UTC m=+111.757154367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9ssv9" (UID: "114b9a1c-0321-4b1d-a570-97c10de99789") : secret "insights-runtime-extractor-tls" not found Jul 08 09:08:26.615706 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:26.615659 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:26.615706 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:26.615720 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:26.616157 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:26.615891 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:42.615868607 +0000 UTC m=+120.865827868 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : configmap references non-existent config key: service-ca.crt Jul 08 09:08:26.616157 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:26.615910 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 09:08:26.616157 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:08:26.615955 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs podName:57259205-640d-4f5f-9e76-a607791fbe62 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:42.615943961 +0000 UTC m=+120.865903264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs") pod "router-default-db9df69b9-6wcdw" (UID: "57259205-640d-4f5f-9e76-a607791fbe62") : secret "router-metrics-certs-default" not found Jul 08 09:08:33.572265 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:33.572229 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:33.572265 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:33.572266 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:33.572845 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:33.572813 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/114b9a1c-0321-4b1d-a570-97c10de99789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:33.574517 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:33.574495 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/114b9a1c-0321-4b1d-a570-97c10de99789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ssv9\" (UID: \"114b9a1c-0321-4b1d-a570-97c10de99789\") " pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:33.667898 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:33.667850 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9ssv9" Jul 08 09:08:33.798690 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:33.798657 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9ssv9"] Jul 08 09:08:33.802281 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:08:33.802250 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114b9a1c_0321_4b1d_a570_97c10de99789.slice/crio-4c23256f904947209318f041a1d2d3c3cae8203a66ff25553e68f18ba4720faa WatchSource:0}: Error finding container 4c23256f904947209318f041a1d2d3c3cae8203a66ff25553e68f18ba4720faa: Status 404 returned error can't find the container with id 4c23256f904947209318f041a1d2d3c3cae8203a66ff25553e68f18ba4720faa Jul 08 09:08:34.701533 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:34.701472 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9ssv9" event={"ID":"114b9a1c-0321-4b1d-a570-97c10de99789","Type":"ContainerStarted","Data":"4c23256f904947209318f041a1d2d3c3cae8203a66ff25553e68f18ba4720faa"} Jul 08 09:08:37.710967 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:37.710882 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9ssv9" event={"ID":"114b9a1c-0321-4b1d-a570-97c10de99789","Type":"ContainerStarted","Data":"a62c712b9d6453943f20efd8836cae32abb96a42d9953b3c3b97903d54e7997e"} Jul 08 09:08:38.715780 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:38.715719 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9ssv9" event={"ID":"114b9a1c-0321-4b1d-a570-97c10de99789","Type":"ContainerStarted","Data":"81fddcdc0fd8b8973c0c79586250b6088cf90f152f535e65bff996796d2065d0"} Jul 08 09:08:40.723443 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:40.723357 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9ssv9" event={"ID":"114b9a1c-0321-4b1d-a570-97c10de99789","Type":"ContainerStarted","Data":"99ab8a3b22f4fe85ec296fbeca15b499493ec6353fa010b867d0e5049d1d7313"} Jul 08 09:08:42.640330 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:42.640287 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:42.640732 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:42.640390 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:42.640953 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:42.640930 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57259205-640d-4f5f-9e76-a607791fbe62-service-ca-bundle\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:42.642644 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:42.642626 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57259205-640d-4f5f-9e76-a607791fbe62-metrics-certs\") pod \"router-default-db9df69b9-6wcdw\" (UID: \"57259205-640d-4f5f-9e76-a607791fbe62\") " pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:42.856361 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:42.856331 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gbk4j\"" Jul 08 09:08:42.864755 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:42.864732 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:42.995769 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:42.995701 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9ssv9" podStartSLOduration=19.457999327 podStartE2EDuration="25.995683483s" totalImagesPullingTime="6.537684156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:08:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:40.751907351 +0000 UTC m=+119.001866635" watchObservedRunningTime="2026-07-08 09:08:42.995683483 +0000 UTC m=+121.245642766" Jul 08 09:08:42.996272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:42.996255 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-db9df69b9-6wcdw"] Jul 08 09:08:42.998631 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:08:42.998607 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57259205_640d_4f5f_9e76_a607791fbe62.slice/crio-4ec38330b8469096f7c1c5c8a81ebb79dcee854957e6fa17af1ef719da3879f1 WatchSource:0}: Error finding container 4ec38330b8469096f7c1c5c8a81ebb79dcee854957e6fa17af1ef719da3879f1: Status 404 returned error can't find the container with id 4ec38330b8469096f7c1c5c8a81ebb79dcee854957e6fa17af1ef719da3879f1 Jul 08 09:08:43.423019 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.422989 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg"] Jul 08 09:08:43.425382 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.425354 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.429104 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.429083 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 09:08:43.430046 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.430022 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 08 09:08:43.430182 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.430074 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 08 09:08:43.430182 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.430082 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 08 09:08:43.430182 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.430157 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 09:08:43.430182 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.430158 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 08 09:08:43.430472 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.430455 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 09:08:43.446374 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.446342 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lh97\" (UniqueName: \"kubernetes.io/projected/c2f6aaf4-2f54-43d8-833b-b69802b1a721-kube-api-access-9lh97\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.446516 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.446387 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.446516 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.446429 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.446516 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.446514 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-ca\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.446639 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.446535 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-hub\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.446639 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.446557 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f6aaf4-2f54-43d8-833b-b69802b1a721-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.447418 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.447398 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg"] Jul 08 09:08:43.547511 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.547474 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lh97\" (UniqueName: \"kubernetes.io/projected/c2f6aaf4-2f54-43d8-833b-b69802b1a721-kube-api-access-9lh97\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.547511 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.547516 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.547776 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.547556 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.547776 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.547624 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-ca\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.547776 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.547654 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-hub\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.547776 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.547683 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f6aaf4-2f54-43d8-833b-b69802b1a721-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.548477 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.548447 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f6aaf4-2f54-43d8-833b-b69802b1a721-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.550274 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.550254 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.550369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.550309 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-ca\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.550476 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.550457 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-hub\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.550511 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.550484 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2f6aaf4-2f54-43d8-833b-b69802b1a721-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.560077 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.560059 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lh97\" (UniqueName: \"kubernetes.io/projected/c2f6aaf4-2f54-43d8-833b-b69802b1a721-kube-api-access-9lh97\") pod \"cluster-proxy-proxy-agent-6746bf69b-7c9mg\" (UID: \"c2f6aaf4-2f54-43d8-833b-b69802b1a721\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.734197 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.734158 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-db9df69b9-6wcdw" event={"ID":"57259205-640d-4f5f-9e76-a607791fbe62","Type":"ContainerStarted","Data":"cf7a82d25d79e0d6ab1ce3988a013d8d3fa4f7e0093489b4f255b1177b7698d9"} Jul 08 09:08:43.734561 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.734217 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-db9df69b9-6wcdw" event={"ID":"57259205-640d-4f5f-9e76-a607791fbe62","Type":"ContainerStarted","Data":"4ec38330b8469096f7c1c5c8a81ebb79dcee854957e6fa17af1ef719da3879f1"} Jul 08 09:08:43.744142 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.744113 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" Jul 08 09:08:43.760851 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.760789 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-db9df69b9-6wcdw" podStartSLOduration=33.760766207 podStartE2EDuration="33.760766207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:43.760089017 +0000 UTC m=+122.010048300" watchObservedRunningTime="2026-07-08 09:08:43.760766207 +0000 UTC m=+122.010725484" Jul 08 09:08:43.865922 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.865891 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:43.868339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.868318 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:43.898199 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:43.898170 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg"] Jul 08 09:08:43.901798 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:08:43.901772 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2f6aaf4_2f54_43d8_833b_b69802b1a721.slice/crio-2b7909e4db678ffb2fa38d02f3f94a0ceceb354ea7a107b287c3ae9e9c19c676 WatchSource:0}: Error finding container 2b7909e4db678ffb2fa38d02f3f94a0ceceb354ea7a107b287c3ae9e9c19c676: Status 404 returned error can't find the container with id 2b7909e4db678ffb2fa38d02f3f94a0ceceb354ea7a107b287c3ae9e9c19c676 Jul 08 09:08:44.739997 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:44.739947 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" event={"ID":"c2f6aaf4-2f54-43d8-833b-b69802b1a721","Type":"ContainerStarted","Data":"2b7909e4db678ffb2fa38d02f3f94a0ceceb354ea7a107b287c3ae9e9c19c676"} Jul 08 09:08:44.740460 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:44.740430 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:44.741892 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:44.741867 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-db9df69b9-6wcdw" Jul 08 09:08:47.752606 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:47.752569 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" event={"ID":"c2f6aaf4-2f54-43d8-833b-b69802b1a721","Type":"ContainerStarted","Data":"e3bca3eb6ab115948d07dd3470e609c62574d6988f32c72ac85a9f037ef7d964"} Jul 08 09:08:47.752980 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:47.752613 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" event={"ID":"c2f6aaf4-2f54-43d8-833b-b69802b1a721","Type":"ContainerStarted","Data":"b6bdfed552fba2ff6e34013cffb4a64d6a36dabcaee4d7886fa240a9f9f0002e"} Jul 08 09:08:47.752980 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:47.752626 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" event={"ID":"c2f6aaf4-2f54-43d8-833b-b69802b1a721","Type":"ContainerStarted","Data":"b7f8cb282ede55d0eb224988e3e5f7f313f2fe307426cf13a66fef721fd6fabb"} Jul 08 09:08:47.775367 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:47.775318 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6746bf69b-7c9mg" podStartSLOduration=1.330139625 podStartE2EDuration="4.775301826s" totalImagesPullingTime="3.445162201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:47.774670074 +0000 UTC m=+126.024629367" watchObservedRunningTime="2026-07-08 09:08:47.775301826 +0000 UTC m=+126.025261108" Jul 08 09:08:51.110500 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:51.110461 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:08:51.112690 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:51.112666 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9eeaebc-02be-4f69-b34d-6f8135fc5374-metrics-certs\") pod \"network-metrics-daemon-lsqn9\" (UID: \"b9eeaebc-02be-4f69-b34d-6f8135fc5374\") " pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:08:51.229074 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:51.229037 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6bnkm\"" Jul 08 09:08:51.236584 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:51.236566 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsqn9" Jul 08 09:08:51.360246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:51.360183 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lsqn9"] Jul 08 09:08:51.362644 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:08:51.362616 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9eeaebc_02be_4f69_b34d_6f8135fc5374.slice/crio-fc0a179c74a46b58c527e7d2b667c693f44edcf9e56e5aaf4f68a0db6917a698 WatchSource:0}: Error finding container fc0a179c74a46b58c527e7d2b667c693f44edcf9e56e5aaf4f68a0db6917a698: Status 404 returned error can't find the container with id fc0a179c74a46b58c527e7d2b667c693f44edcf9e56e5aaf4f68a0db6917a698 Jul 08 09:08:51.766528 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:51.766491 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lsqn9" event={"ID":"b9eeaebc-02be-4f69-b34d-6f8135fc5374","Type":"ContainerStarted","Data":"fc0a179c74a46b58c527e7d2b667c693f44edcf9e56e5aaf4f68a0db6917a698"} Jul 08 09:08:52.771475 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:52.771440 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lsqn9" event={"ID":"b9eeaebc-02be-4f69-b34d-6f8135fc5374","Type":"ContainerStarted","Data":"32b4e3f0f5e0c30f9a67a9b1d5ea3c537e2d59a02893f203fc32953fa35233d5"} Jul 08 09:08:52.771864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:52.771503 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lsqn9" event={"ID":"b9eeaebc-02be-4f69-b34d-6f8135fc5374","Type":"ContainerStarted","Data":"ee8084b89b162593df1bf29cd2273fb207a701f83e28c98ea12b53ac1a5bcb96"} Jul 08 09:08:52.800218 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:52.800167 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lsqn9" podStartSLOduration=129.960242279 podStartE2EDuration="2m10.800152101s" totalImagesPullingTime="839.909822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:52.798299688 +0000 UTC m=+131.048258985" watchObservedRunningTime="2026-07-08 09:08:52.800152101 +0000 UTC m=+131.050111381" Jul 08 09:08:54.934177 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:54.934136 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d6zkz"] Jul 08 09:08:54.937691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:54.937663 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:54.941623 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:54.941603 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 09:08:54.941736 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:54.941632 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 09:08:54.943087 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:54.943069 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 09:08:54.943180 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:54.943088 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tzxvj\"" Jul 08 09:08:54.943180 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:54.943134 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 09:08:54.944310 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:54.944291 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 09:08:54.944523 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:54.944510 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 09:08:55.041384 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.041349 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x5bh\" (UniqueName: \"kubernetes.io/projected/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-kube-api-access-8x5bh\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.041384 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.041387 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-textfile\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.041598 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.041418 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-wtmp\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.041598 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.041440 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-metrics-client-ca\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.041598 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.041498 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.041598 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.041579 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-root\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.041723 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.041621 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-sys\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.041723 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.041649 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.041723 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.041672 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-tls\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.142929 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.142887 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.142929 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.142925 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-tls\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143154 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.142969 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x5bh\" (UniqueName: \"kubernetes.io/projected/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-kube-api-access-8x5bh\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143154 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.142993 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-textfile\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143154 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143013 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-wtmp\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143154 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143028 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-metrics-client-ca\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143154 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143044 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143154 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143064 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-root\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143154 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143113 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-sys\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143472 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143173 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-sys\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143472 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143207 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-wtmp\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143472 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143229 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-root\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143472 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143391 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-textfile\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143644 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143618 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.143699 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.143679 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-metrics-client-ca\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.145393 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.145354 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.145393 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.145388 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-node-exporter-tls\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.153426 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.153399 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x5bh\" (UniqueName: \"kubernetes.io/projected/6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd-kube-api-access-8x5bh\") pod \"node-exporter-d6zkz\" (UID: \"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd\") " pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.246155 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.246124 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d6zkz" Jul 08 09:08:55.256912 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:08:55.256883 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a8ad7f6_9add_4f66_a2b5_179a3cfdcadd.slice/crio-155b3a8338790c057c2a6331b72bf9a2ae0fd3e1ba21355ddebcdd99a51560a2 WatchSource:0}: Error finding container 155b3a8338790c057c2a6331b72bf9a2ae0fd3e1ba21355ddebcdd99a51560a2: Status 404 returned error can't find the container with id 155b3a8338790c057c2a6331b72bf9a2ae0fd3e1ba21355ddebcdd99a51560a2 Jul 08 09:08:55.786325 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:55.786296 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6zkz" event={"ID":"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd","Type":"ContainerStarted","Data":"155b3a8338790c057c2a6331b72bf9a2ae0fd3e1ba21355ddebcdd99a51560a2"} Jul 08 09:08:57.794682 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:57.794643 2572 generic.go:356] "Generic (PLEG): container finished" podID="6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd" containerID="3180a0f07f80d4240ac32af60c4900156e6c155d31d6b62ee6069a5ca56b1343" exitCode=0 Jul 08 09:08:57.795275 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:57.794695 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6zkz" event={"ID":"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd","Type":"ContainerDied","Data":"3180a0f07f80d4240ac32af60c4900156e6c155d31d6b62ee6069a5ca56b1343"} Jul 08 09:08:58.799357 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:58.799317 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6zkz" event={"ID":"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd","Type":"ContainerStarted","Data":"d0b055ed0f91efc856294509a82335fffcbf047047e97a59a81048684e4eafc3"} Jul 08 09:08:58.799730 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:58.799380 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6zkz" event={"ID":"6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd","Type":"ContainerStarted","Data":"fc9cb5f8621f49c89b1b558ef6470f360287d64d817fc1719d9e91b611cd0a5e"} Jul 08 09:08:58.839444 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:08:58.839393 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d6zkz" podStartSLOduration=3.238519231 podStartE2EDuration="4.839378656s" totalImagesPullingTime="1.537990924s" totalInitContainerRuntime="62.868501ms" isStatefulPod=true podCreationTimestamp="2026-07-08 09:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:58.837787515 +0000 UTC m=+137.087746799" watchObservedRunningTime="2026-07-08 09:08:58.839378656 +0000 UTC m=+137.089337994" Jul 08 09:09:05.728463 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.728427 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-676bb5bbc6-xjfxl"] Jul 08 09:09:05.728874 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:09:05.728629 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" podUID="d8c644b8-b197-4382-8e5f-806cdc3323f5" Jul 08 09:09:05.819995 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.819967 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:09:05.830782 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.830738 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:09:05.930025 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.929976 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-bound-sa-token\") pod \"d8c644b8-b197-4382-8e5f-806cdc3323f5\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " Jul 08 09:09:05.930025 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.930031 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d8c644b8-b197-4382-8e5f-806cdc3323f5-ca-trust-extracted\") pod \"d8c644b8-b197-4382-8e5f-806cdc3323f5\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " Jul 08 09:09:05.930280 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.930069 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-certificates\") pod \"d8c644b8-b197-4382-8e5f-806cdc3323f5\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " Jul 08 09:09:05.930280 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.930100 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-image-registry-private-configuration\") pod \"d8c644b8-b197-4382-8e5f-806cdc3323f5\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " Jul 08 09:09:05.930280 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.930143 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8dnt\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-kube-api-access-q8dnt\") pod \"d8c644b8-b197-4382-8e5f-806cdc3323f5\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " Jul 08 09:09:05.930280 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.930177 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-trusted-ca\") pod \"d8c644b8-b197-4382-8e5f-806cdc3323f5\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " Jul 08 09:09:05.930280 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.930221 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-installation-pull-secrets\") pod \"d8c644b8-b197-4382-8e5f-806cdc3323f5\" (UID: \"d8c644b8-b197-4382-8e5f-806cdc3323f5\") " Jul 08 09:09:05.930505 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.930354 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c644b8-b197-4382-8e5f-806cdc3323f5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d8c644b8-b197-4382-8e5f-806cdc3323f5" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:09:05.930505 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.930430 2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d8c644b8-b197-4382-8e5f-806cdc3323f5-ca-trust-extracted\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:09:05.930612 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.930444 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d8c644b8-b197-4382-8e5f-806cdc3323f5" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:09:05.930668 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.930630 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d8c644b8-b197-4382-8e5f-806cdc3323f5" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:09:05.932350 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.932312 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d8c644b8-b197-4382-8e5f-806cdc3323f5" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:09:05.932469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.932361 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d8c644b8-b197-4382-8e5f-806cdc3323f5" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:09:05.932469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.932411 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-kube-api-access-q8dnt" (OuterVolumeSpecName: "kube-api-access-q8dnt") pod "d8c644b8-b197-4382-8e5f-806cdc3323f5" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5"). InnerVolumeSpecName "kube-api-access-q8dnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:09:05.932469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:05.932432 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d8c644b8-b197-4382-8e5f-806cdc3323f5" (UID: "d8c644b8-b197-4382-8e5f-806cdc3323f5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:09:06.031567 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:06.031521 2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-bound-sa-token\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:09:06.031567 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:06.031565 2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-certificates\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:09:06.031567 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:06.031576 2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-image-registry-private-configuration\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:09:06.031567 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:06.031585 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8dnt\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-kube-api-access-q8dnt\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:09:06.031864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:06.031594 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8c644b8-b197-4382-8e5f-806cdc3323f5-trusted-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:09:06.031864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:06.031604 2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d8c644b8-b197-4382-8e5f-806cdc3323f5-installation-pull-secrets\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:09:06.823235 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:06.823204 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676bb5bbc6-xjfxl" Jul 08 09:09:06.859385 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:06.859355 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-676bb5bbc6-xjfxl"] Jul 08 09:09:06.865905 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:06.865880 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-676bb5bbc6-xjfxl"] Jul 08 09:09:06.940824 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:06.940786 2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8c644b8-b197-4382-8e5f-806cdc3323f5-registry-tls\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:09:08.306258 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:08.306226 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8c644b8-b197-4382-8e5f-806cdc3323f5" path="/var/lib/kubelet/pods/d8c644b8-b197-4382-8e5f-806cdc3323f5/volumes" Jul 08 09:09:09.007022 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.006983 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66dc5488d6-vkh2t"] Jul 08 09:09:09.011451 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.011426 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.015805 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.015783 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 09:09:09.016252 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.016235 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 09:09:09.016321 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.016243 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 09:09:09.016321 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.016297 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 09:09:09.017141 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.017121 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 09:09:09.017251 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.017166 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 09:09:09.017251 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.017174 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-k7z6k\"" Jul 08 09:09:09.017251 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.017174 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 09:09:09.025069 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.025052 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 09:09:09.035709 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.035692 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66dc5488d6-vkh2t"] Jul 08 09:09:09.158149 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.158105 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-console-config\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.158149 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.158155 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-oauth-config\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.158396 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.158204 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-service-ca\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.158396 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.158228 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrz9b\" (UniqueName: \"kubernetes.io/projected/8c6edf6d-fa36-407f-a158-61e8abccce30-kube-api-access-rrz9b\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.158396 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.158246 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-serving-cert\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.158396 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.158264 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-trusted-ca-bundle\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.158396 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.158286 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-oauth-serving-cert\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.259121 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.259031 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-service-ca\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.259121 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.259080 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrz9b\" (UniqueName: \"kubernetes.io/projected/8c6edf6d-fa36-407f-a158-61e8abccce30-kube-api-access-rrz9b\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.259121 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.259100 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-serving-cert\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.259121 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.259121 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-trusted-ca-bundle\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.259500 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.259153 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-oauth-serving-cert\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.259500 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.259221 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-console-config\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.259924 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.259888 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-oauth-config\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.260498 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.260468 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-service-ca\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.260605 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.260381 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-oauth-serving-cert\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.260804 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.260776 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-trusted-ca-bundle\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.264691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.260889 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-console-config\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.264691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.262788 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-serving-cert\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.264691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.263484 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-oauth-config\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.269988 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.269968 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrz9b\" (UniqueName: \"kubernetes.io/projected/8c6edf6d-fa36-407f-a158-61e8abccce30-kube-api-access-rrz9b\") pod \"console-66dc5488d6-vkh2t\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.321091 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.321050 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:09.449511 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.449477 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66dc5488d6-vkh2t"] Jul 08 09:09:09.452539 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:09:09.452509 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6edf6d_fa36_407f_a158_61e8abccce30.slice/crio-d383bbfb8afe480f56a130f3b45911f2bb2c31d95cdfc8bdf050f11eaf8f5c56 WatchSource:0}: Error finding container d383bbfb8afe480f56a130f3b45911f2bb2c31d95cdfc8bdf050f11eaf8f5c56: Status 404 returned error can't find the container with id d383bbfb8afe480f56a130f3b45911f2bb2c31d95cdfc8bdf050f11eaf8f5c56 Jul 08 09:09:09.834163 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:09.834126 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66dc5488d6-vkh2t" event={"ID":"8c6edf6d-fa36-407f-a158-61e8abccce30","Type":"ContainerStarted","Data":"d383bbfb8afe480f56a130f3b45911f2bb2c31d95cdfc8bdf050f11eaf8f5c56"} Jul 08 09:09:12.845247 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:12.844997 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66dc5488d6-vkh2t" event={"ID":"8c6edf6d-fa36-407f-a158-61e8abccce30","Type":"ContainerStarted","Data":"1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa"} Jul 08 09:09:12.879783 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:12.879710 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66dc5488d6-vkh2t" podStartSLOduration=1.600684129 podStartE2EDuration="4.879687854s" totalImagesPullingTime="3.279003725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:09:12.878685197 +0000 UTC m=+151.128644476" watchObservedRunningTime="2026-07-08 09:09:12.879687854 +0000 UTC m=+151.129647137" Jul 08 09:09:19.322240 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:19.322204 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:19.322682 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:19.322266 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:19.326952 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:19.326932 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:19.620770 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:09:19.620655 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zg4rz" podUID="be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8" Jul 08 09:09:19.867118 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:19.867084 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zg4rz" Jul 08 09:09:19.871005 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:19.870949 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:09:24.590347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:24.590311 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:09:24.592570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:24.592549 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8-metrics-tls\") pod \"dns-default-zg4rz\" (UID: \"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8\") " pod="openshift-dns/dns-default-zg4rz" Jul 08 09:09:24.671201 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:24.671173 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qglp2\"" Jul 08 09:09:24.678121 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:24.678100 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zg4rz" Jul 08 09:09:24.812566 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:24.812544 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zg4rz"] Jul 08 09:09:24.814914 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:09:24.814888 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2b4a12_517d_4fe5_9ca3_24e1fa3bb7d8.slice/crio-0c2117b57256c70dbdc3b8ac4833bf94ffd4fc63a33096fa547863caa326d98d WatchSource:0}: Error finding container 0c2117b57256c70dbdc3b8ac4833bf94ffd4fc63a33096fa547863caa326d98d: Status 404 returned error can't find the container with id 0c2117b57256c70dbdc3b8ac4833bf94ffd4fc63a33096fa547863caa326d98d Jul 08 09:09:24.883721 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:24.883636 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zg4rz" event={"ID":"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8","Type":"ContainerStarted","Data":"0c2117b57256c70dbdc3b8ac4833bf94ffd4fc63a33096fa547863caa326d98d"} Jul 08 09:09:27.895323 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:27.895290 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zg4rz" event={"ID":"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8","Type":"ContainerStarted","Data":"797c492c1a1d39101e4f783bf10fa22da49e53d43c906dd1d723dfd9b9afd0be"} Jul 08 09:09:27.895718 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:27.895338 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zg4rz" event={"ID":"be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8","Type":"ContainerStarted","Data":"dd9a000caecd9068b1cd1d58d6dc9a829ae49b8062e341898119ea2f033ee65e"} Jul 08 09:09:27.895718 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:27.895418 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zg4rz" Jul 08 09:09:27.916284 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:27.916236 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zg4rz" podStartSLOduration=129.680474817 podStartE2EDuration="2m11.916221486s" totalImagesPullingTime="2.235746669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:09:27.915233447 +0000 UTC m=+166.165192732" watchObservedRunningTime="2026-07-08 09:09:27.916221486 +0000 UTC m=+166.166180768" Jul 08 09:09:37.901832 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:09:37.901707 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zg4rz" Jul 08 09:10:19.569610 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.569577 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6868869c78-5bsvt"] Jul 08 09:10:19.572902 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.572885 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.577914 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.577891 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 09:10:19.578266 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.578218 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 09:10:19.578470 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.578454 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 09:10:19.578561 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.578437 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 09:10:19.578561 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.578458 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 09:10:19.578661 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.578458 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rjfx5\"" Jul 08 09:10:19.605426 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.605392 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 09:10:19.605902 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.605883 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6868869c78-5bsvt"] Jul 08 09:10:19.686978 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.686942 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78f7a695-a225-4296-8e26-3cce33373aa8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.686978 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.686979 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-secret-telemeter-client\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.687239 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.687002 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.687239 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.687022 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78f7a695-a225-4296-8e26-3cce33373aa8-serving-certs-ca-bundle\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.687239 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.687101 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-telemeter-client-tls\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.687239 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.687164 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-federate-client-tls\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.687239 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.687192 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78f7a695-a225-4296-8e26-3cce33373aa8-metrics-client-ca\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.687239 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.687219 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztfb9\" (UniqueName: \"kubernetes.io/projected/78f7a695-a225-4296-8e26-3cce33373aa8-kube-api-access-ztfb9\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.788595 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.788545 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78f7a695-a225-4296-8e26-3cce33373aa8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.788595 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.788602 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-secret-telemeter-client\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.788878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.788623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.788878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.788737 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78f7a695-a225-4296-8e26-3cce33373aa8-serving-certs-ca-bundle\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.788878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.788809 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-telemeter-client-tls\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.788878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.788836 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-federate-client-tls\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.788878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.788857 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78f7a695-a225-4296-8e26-3cce33373aa8-metrics-client-ca\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.788878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.788873 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztfb9\" (UniqueName: \"kubernetes.io/projected/78f7a695-a225-4296-8e26-3cce33373aa8-kube-api-access-ztfb9\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.789571 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.789544 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78f7a695-a225-4296-8e26-3cce33373aa8-serving-certs-ca-bundle\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.789707 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.789588 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78f7a695-a225-4296-8e26-3cce33373aa8-metrics-client-ca\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.789887 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.789863 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78f7a695-a225-4296-8e26-3cce33373aa8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.791363 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.791338 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-secret-telemeter-client\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.791554 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.791537 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.791609 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.791544 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-telemeter-client-tls\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.791717 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.791695 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/78f7a695-a225-4296-8e26-3cce33373aa8-federate-client-tls\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.797761 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.797727 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztfb9\" (UniqueName: \"kubernetes.io/projected/78f7a695-a225-4296-8e26-3cce33373aa8-kube-api-access-ztfb9\") pod \"telemeter-client-6868869c78-5bsvt\" (UID: \"78f7a695-a225-4296-8e26-3cce33373aa8\") " pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:19.881920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:19.881830 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" Jul 08 09:10:20.015103 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:20.015070 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6868869c78-5bsvt"] Jul 08 09:10:20.018983 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:10:20.018954 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f7a695_a225_4296_8e26_3cce33373aa8.slice/crio-18698bde62afede651d2da44e4dff600b573a36ab94d4a12c2ed6d4e00ef00e3 WatchSource:0}: Error finding container 18698bde62afede651d2da44e4dff600b573a36ab94d4a12c2ed6d4e00ef00e3: Status 404 returned error can't find the container with id 18698bde62afede651d2da44e4dff600b573a36ab94d4a12c2ed6d4e00ef00e3 Jul 08 09:10:20.062242 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:20.062212 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" event={"ID":"78f7a695-a225-4296-8e26-3cce33373aa8","Type":"ContainerStarted","Data":"18698bde62afede651d2da44e4dff600b573a36ab94d4a12c2ed6d4e00ef00e3"} Jul 08 09:10:23.075526 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:23.075435 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" event={"ID":"78f7a695-a225-4296-8e26-3cce33373aa8","Type":"ContainerStarted","Data":"e32aa3dd68654954dd72e8df10e7ae9de02804d7f07f1885c01069d795d57af3"} Jul 08 09:10:25.086123 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.086082 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" event={"ID":"78f7a695-a225-4296-8e26-3cce33373aa8","Type":"ContainerStarted","Data":"69cd7f48eda0844350b337a88d7891186f7df266d552da4dc2fe4c9f99b1fc03"} Jul 08 09:10:25.086699 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.086142 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" event={"ID":"78f7a695-a225-4296-8e26-3cce33373aa8","Type":"ContainerStarted","Data":"26d9b5144911ca5b46bf9d490059917ad34604a094d5a8206990bfe26e141bf0"} Jul 08 09:10:25.111142 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.111055 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6868869c78-5bsvt" podStartSLOduration=2.050873155 podStartE2EDuration="6.111036216s" totalImagesPullingTime="4.060163061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:10:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:10:25.110280902 +0000 UTC m=+223.360240186" watchObservedRunningTime="2026-07-08 09:10:25.111036216 +0000 UTC m=+223.360995500" Jul 08 09:10:25.788917 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.788886 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68794d9fd4-9dk7d"] Jul 08 09:10:25.792033 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.792010 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:25.804224 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.804199 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68794d9fd4-9dk7d"] Jul 08 09:10:25.938815 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.938769 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-console-config\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:25.938815 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.938820 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-service-ca\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:25.939049 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.938843 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-trusted-ca-bundle\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:25.939049 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.938934 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8gs9\" (UniqueName: \"kubernetes.io/projected/6109e4ad-2987-4818-918c-9b754adcaffe-kube-api-access-q8gs9\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:25.939049 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.938982 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-serving-cert\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:25.939049 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.938999 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-oauth-serving-cert\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:25.939049 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:25.939039 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-oauth-config\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.039805 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.039696 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-oauth-config\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.039805 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.039739 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-console-config\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.039805 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.039778 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-service-ca\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.039805 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.039805 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-trusted-ca-bundle\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.040103 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.039843 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8gs9\" (UniqueName: \"kubernetes.io/projected/6109e4ad-2987-4818-918c-9b754adcaffe-kube-api-access-q8gs9\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.040103 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.039890 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-serving-cert\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.040103 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.039913 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-oauth-serving-cert\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.040582 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.040551 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-service-ca\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.040728 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.040702 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-oauth-serving-cert\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.040834 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.040702 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-console-config\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.040834 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.040799 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-trusted-ca-bundle\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.042301 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.042276 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-serving-cert\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.042396 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.042380 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-oauth-config\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.050009 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.049991 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8gs9\" (UniqueName: \"kubernetes.io/projected/6109e4ad-2987-4818-918c-9b754adcaffe-kube-api-access-q8gs9\") pod \"console-68794d9fd4-9dk7d\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.100963 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.100925 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:26.234059 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:26.234036 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68794d9fd4-9dk7d"] Jul 08 09:10:26.236198 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:10:26.236178 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6109e4ad_2987_4818_918c_9b754adcaffe.slice/crio-b5662573eff4a456db09a90a23494ec25716227fe6406e5c1ab27cdcfcd21556 WatchSource:0}: Error finding container b5662573eff4a456db09a90a23494ec25716227fe6406e5c1ab27cdcfcd21556: Status 404 returned error can't find the container with id b5662573eff4a456db09a90a23494ec25716227fe6406e5c1ab27cdcfcd21556 Jul 08 09:10:27.096433 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:27.096395 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68794d9fd4-9dk7d" event={"ID":"6109e4ad-2987-4818-918c-9b754adcaffe","Type":"ContainerStarted","Data":"f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3"} Jul 08 09:10:27.096601 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:27.096452 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68794d9fd4-9dk7d" event={"ID":"6109e4ad-2987-4818-918c-9b754adcaffe","Type":"ContainerStarted","Data":"b5662573eff4a456db09a90a23494ec25716227fe6406e5c1ab27cdcfcd21556"} Jul 08 09:10:27.123483 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:27.123430 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68794d9fd4-9dk7d" podStartSLOduration=2.12341254 podStartE2EDuration="2.12341254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:10:27.12195507 +0000 UTC m=+225.371914353" watchObservedRunningTime="2026-07-08 09:10:27.12341254 +0000 UTC m=+225.373371852" Jul 08 09:10:36.102032 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:36.101985 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:36.102634 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:36.102060 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:36.106660 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:36.106642 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:36.131768 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:36.131728 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:10:36.219522 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:10:36.219427 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66dc5488d6-vkh2t"] Jul 08 09:11:01.244025 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.243972 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-66dc5488d6-vkh2t" podUID="8c6edf6d-fa36-407f-a158-61e8abccce30" containerName="console" containerID="cri-o://1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa" gracePeriod=15 Jul 08 09:11:01.497283 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.497217 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:11:01.592626 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.592586 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-oauth-config\") pod \"8c6edf6d-fa36-407f-a158-61e8abccce30\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " Jul 08 09:11:01.592626 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.592630 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-service-ca\") pod \"8c6edf6d-fa36-407f-a158-61e8abccce30\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " Jul 08 09:11:01.592915 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.592674 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-console-config\") pod \"8c6edf6d-fa36-407f-a158-61e8abccce30\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " Jul 08 09:11:01.592915 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.592692 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-oauth-serving-cert\") pod \"8c6edf6d-fa36-407f-a158-61e8abccce30\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " Jul 08 09:11:01.592915 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.592719 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrz9b\" (UniqueName: \"kubernetes.io/projected/8c6edf6d-fa36-407f-a158-61e8abccce30-kube-api-access-rrz9b\") pod \"8c6edf6d-fa36-407f-a158-61e8abccce30\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " Jul 08 09:11:01.592915 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.592775 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-trusted-ca-bundle\") pod \"8c6edf6d-fa36-407f-a158-61e8abccce30\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " Jul 08 09:11:01.592915 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.592818 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-serving-cert\") pod \"8c6edf6d-fa36-407f-a158-61e8abccce30\" (UID: \"8c6edf6d-fa36-407f-a158-61e8abccce30\") " Jul 08 09:11:01.593264 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.593237 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-service-ca" (OuterVolumeSpecName: "service-ca") pod "8c6edf6d-fa36-407f-a158-61e8abccce30" (UID: "8c6edf6d-fa36-407f-a158-61e8abccce30"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:11:01.593327 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.593241 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-console-config" (OuterVolumeSpecName: "console-config") pod "8c6edf6d-fa36-407f-a158-61e8abccce30" (UID: "8c6edf6d-fa36-407f-a158-61e8abccce30"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:11:01.593327 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.593306 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8c6edf6d-fa36-407f-a158-61e8abccce30" (UID: "8c6edf6d-fa36-407f-a158-61e8abccce30"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:11:01.593430 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.593365 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8c6edf6d-fa36-407f-a158-61e8abccce30" (UID: "8c6edf6d-fa36-407f-a158-61e8abccce30"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:11:01.595064 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.595037 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c6edf6d-fa36-407f-a158-61e8abccce30-kube-api-access-rrz9b" (OuterVolumeSpecName: "kube-api-access-rrz9b") pod "8c6edf6d-fa36-407f-a158-61e8abccce30" (UID: "8c6edf6d-fa36-407f-a158-61e8abccce30"). InnerVolumeSpecName "kube-api-access-rrz9b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:11:01.595165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.595094 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8c6edf6d-fa36-407f-a158-61e8abccce30" (UID: "8c6edf6d-fa36-407f-a158-61e8abccce30"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:11:01.595165 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.595124 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8c6edf6d-fa36-407f-a158-61e8abccce30" (UID: "8c6edf6d-fa36-407f-a158-61e8abccce30"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:11:01.693599 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.693550 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-oauth-config\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:11:01.693599 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.693592 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-service-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:11:01.693599 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.693602 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-console-config\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:11:01.693599 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.693611 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-oauth-serving-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:11:01.693599 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.693620 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrz9b\" (UniqueName: \"kubernetes.io/projected/8c6edf6d-fa36-407f-a158-61e8abccce30-kube-api-access-rrz9b\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:11:01.693912 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.693631 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6edf6d-fa36-407f-a158-61e8abccce30-trusted-ca-bundle\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:11:01.693912 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:01.693640 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c6edf6d-fa36-407f-a158-61e8abccce30-console-serving-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:11:02.214400 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:02.214365 2572 generic.go:356] "Generic (PLEG): container finished" podID="8c6edf6d-fa36-407f-a158-61e8abccce30" containerID="1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa" exitCode=0 Jul 08 09:11:02.214570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:02.214412 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66dc5488d6-vkh2t" Jul 08 09:11:02.214570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:02.214424 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66dc5488d6-vkh2t" event={"ID":"8c6edf6d-fa36-407f-a158-61e8abccce30","Type":"ContainerDied","Data":"1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa"} Jul 08 09:11:02.214570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:02.214464 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66dc5488d6-vkh2t" event={"ID":"8c6edf6d-fa36-407f-a158-61e8abccce30","Type":"ContainerDied","Data":"d383bbfb8afe480f56a130f3b45911f2bb2c31d95cdfc8bdf050f11eaf8f5c56"} Jul 08 09:11:02.214570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:02.214483 2572 scope.go:117] "RemoveContainer" containerID="1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa" Jul 08 09:11:02.223015 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:02.222996 2572 scope.go:117] "RemoveContainer" containerID="1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa" Jul 08 09:11:02.223269 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:11:02.223252 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa\": container with ID starting with 1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa not found: ID does not exist" containerID="1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa" Jul 08 09:11:02.223318 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:02.223276 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa"} err="failed to get container status \"1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa\": rpc error: code = NotFound desc = could not find container \"1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa\": container with ID starting with 1f74fe7fb604e6a6781e6dfcc8bb015380a70b425d87973f3adbf67c3f65d9aa not found: ID does not exist" Jul 08 09:11:02.240191 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:02.240164 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66dc5488d6-vkh2t"] Jul 08 09:11:02.247119 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:02.247096 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66dc5488d6-vkh2t"] Jul 08 09:11:02.306879 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:02.306849 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c6edf6d-fa36-407f-a158-61e8abccce30" path="/var/lib/kubelet/pods/8c6edf6d-fa36-407f-a158-61e8abccce30/volumes" Jul 08 09:11:42.202917 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:11:42.202890 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 09:12:04.472042 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:04.472011 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68794d9fd4-9dk7d"] Jul 08 09:12:29.492947 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.492881 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-68794d9fd4-9dk7d" podUID="6109e4ad-2987-4818-918c-9b754adcaffe" containerName="console" containerID="cri-o://f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3" gracePeriod=15 Jul 08 09:12:29.728274 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.728250 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:12:29.730536 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.730515 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-console-config\") pod \"6109e4ad-2987-4818-918c-9b754adcaffe\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " Jul 08 09:12:29.730599 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.730545 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-service-ca\") pod \"6109e4ad-2987-4818-918c-9b754adcaffe\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " Jul 08 09:12:29.730599 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.730566 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-serving-cert\") pod \"6109e4ad-2987-4818-918c-9b754adcaffe\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " Jul 08 09:12:29.730599 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.730591 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-oauth-config\") pod \"6109e4ad-2987-4818-918c-9b754adcaffe\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " Jul 08 09:12:29.730776 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.730711 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-oauth-serving-cert\") pod \"6109e4ad-2987-4818-918c-9b754adcaffe\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " Jul 08 09:12:29.730839 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.730797 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-trusted-ca-bundle\") pod \"6109e4ad-2987-4818-918c-9b754adcaffe\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " Jul 08 09:12:29.730839 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.730814 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8gs9\" (UniqueName: \"kubernetes.io/projected/6109e4ad-2987-4818-918c-9b754adcaffe-kube-api-access-q8gs9\") pod \"6109e4ad-2987-4818-918c-9b754adcaffe\" (UID: \"6109e4ad-2987-4818-918c-9b754adcaffe\") " Jul 08 09:12:29.730938 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.730907 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-console-config" (OuterVolumeSpecName: "console-config") pod "6109e4ad-2987-4818-918c-9b754adcaffe" (UID: "6109e4ad-2987-4818-918c-9b754adcaffe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:12:29.731107 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.731081 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-service-ca" (OuterVolumeSpecName: "service-ca") pod "6109e4ad-2987-4818-918c-9b754adcaffe" (UID: "6109e4ad-2987-4818-918c-9b754adcaffe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:12:29.731275 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.731232 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6109e4ad-2987-4818-918c-9b754adcaffe" (UID: "6109e4ad-2987-4818-918c-9b754adcaffe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:12:29.731375 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.731355 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6109e4ad-2987-4818-918c-9b754adcaffe" (UID: "6109e4ad-2987-4818-918c-9b754adcaffe"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:12:29.732672 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.732648 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6109e4ad-2987-4818-918c-9b754adcaffe" (UID: "6109e4ad-2987-4818-918c-9b754adcaffe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:12:29.733060 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.733038 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6109e4ad-2987-4818-918c-9b754adcaffe" (UID: "6109e4ad-2987-4818-918c-9b754adcaffe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:12:29.733151 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.733121 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6109e4ad-2987-4818-918c-9b754adcaffe-kube-api-access-q8gs9" (OuterVolumeSpecName: "kube-api-access-q8gs9") pod "6109e4ad-2987-4818-918c-9b754adcaffe" (UID: "6109e4ad-2987-4818-918c-9b754adcaffe"). InnerVolumeSpecName "kube-api-access-q8gs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:12:29.831468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.831376 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-trusted-ca-bundle\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:12:29.831468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.831408 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8gs9\" (UniqueName: \"kubernetes.io/projected/6109e4ad-2987-4818-918c-9b754adcaffe-kube-api-access-q8gs9\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:12:29.831468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.831418 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-console-config\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:12:29.831468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.831426 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-service-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:12:29.831468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.831437 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-serving-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:12:29.831468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.831445 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6109e4ad-2987-4818-918c-9b754adcaffe-console-oauth-config\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:12:29.831468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:29.831452 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6109e4ad-2987-4818-918c-9b754adcaffe-oauth-serving-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:12:30.505690 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:30.505657 2572 generic.go:356] "Generic (PLEG): container finished" podID="6109e4ad-2987-4818-918c-9b754adcaffe" containerID="f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3" exitCode=0 Jul 08 09:12:30.506116 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:30.505701 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68794d9fd4-9dk7d" Jul 08 09:12:30.506116 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:30.505709 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68794d9fd4-9dk7d" event={"ID":"6109e4ad-2987-4818-918c-9b754adcaffe","Type":"ContainerDied","Data":"f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3"} Jul 08 09:12:30.506116 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:30.505768 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68794d9fd4-9dk7d" event={"ID":"6109e4ad-2987-4818-918c-9b754adcaffe","Type":"ContainerDied","Data":"b5662573eff4a456db09a90a23494ec25716227fe6406e5c1ab27cdcfcd21556"} Jul 08 09:12:30.506116 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:30.505789 2572 scope.go:117] "RemoveContainer" containerID="f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3" Jul 08 09:12:30.513332 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:30.513314 2572 scope.go:117] "RemoveContainer" containerID="f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3" Jul 08 09:12:30.513572 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:12:30.513553 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3\": container with ID starting with f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3 not found: ID does not exist" containerID="f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3" Jul 08 09:12:30.513618 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:30.513580 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3"} err="failed to get container status \"f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3\": rpc error: code = NotFound desc = could not find container \"f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3\": container with ID starting with f7a62400e30633f2b4a035cd7424752058ffa139883eb3736ed75a6fbac115b3 not found: ID does not exist" Jul 08 09:12:30.528319 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:30.528287 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68794d9fd4-9dk7d"] Jul 08 09:12:30.533278 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:30.533255 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68794d9fd4-9dk7d"] Jul 08 09:12:31.620276 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.620241 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9"] Jul 08 09:12:31.620646 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.620460 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c6edf6d-fa36-407f-a158-61e8abccce30" containerName="console" Jul 08 09:12:31.620646 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.620472 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c6edf6d-fa36-407f-a158-61e8abccce30" containerName="console" Jul 08 09:12:31.620646 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.620487 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6109e4ad-2987-4818-918c-9b754adcaffe" containerName="console" Jul 08 09:12:31.620646 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.620493 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6109e4ad-2987-4818-918c-9b754adcaffe" containerName="console" Jul 08 09:12:31.620646 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.620540 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c6edf6d-fa36-407f-a158-61e8abccce30" containerName="console" Jul 08 09:12:31.620646 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.620547 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6109e4ad-2987-4818-918c-9b754adcaffe" containerName="console" Jul 08 09:12:31.624492 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.624476 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:31.628368 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.628348 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-c5lh2\"" Jul 08 09:12:31.628521 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.628407 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 09:12:31.629696 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.629677 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 09:12:31.638796 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.638773 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9"] Jul 08 09:12:31.642143 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.642122 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k7v5\" (UniqueName: \"kubernetes.io/projected/6df5644a-f34e-4814-af4e-8598341baf1f-kube-api-access-8k7v5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:31.642256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.642170 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:31.642309 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.642253 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:31.742645 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.742599 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k7v5\" (UniqueName: \"kubernetes.io/projected/6df5644a-f34e-4814-af4e-8598341baf1f-kube-api-access-8k7v5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:31.742645 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.742659 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:31.742900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.742700 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:31.743055 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.743040 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:31.743125 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.743108 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:31.752665 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.752637 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k7v5\" (UniqueName: \"kubernetes.io/projected/6df5644a-f34e-4814-af4e-8598341baf1f-kube-api-access-8k7v5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:31.933348 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:31.933239 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:32.065376 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:32.065353 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9"] Jul 08 09:12:32.067682 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:12:32.067654 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df5644a_f34e_4814_af4e_8598341baf1f.slice/crio-06e9cbf4d87211c7ce68510c8283fa3f19e1ad28332b4a4db99466b4dfdcb64f WatchSource:0}: Error finding container 06e9cbf4d87211c7ce68510c8283fa3f19e1ad28332b4a4db99466b4dfdcb64f: Status 404 returned error can't find the container with id 06e9cbf4d87211c7ce68510c8283fa3f19e1ad28332b4a4db99466b4dfdcb64f Jul 08 09:12:32.069617 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:32.069597 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 09:12:32.306497 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:32.306465 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6109e4ad-2987-4818-918c-9b754adcaffe" path="/var/lib/kubelet/pods/6109e4ad-2987-4818-918c-9b754adcaffe/volumes" Jul 08 09:12:32.513724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:32.513690 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" event={"ID":"6df5644a-f34e-4814-af4e-8598341baf1f","Type":"ContainerStarted","Data":"06e9cbf4d87211c7ce68510c8283fa3f19e1ad28332b4a4db99466b4dfdcb64f"} Jul 08 09:12:40.545541 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:40.545506 2572 generic.go:356] "Generic (PLEG): container finished" podID="6df5644a-f34e-4814-af4e-8598341baf1f" containerID="d9f4711c1f467656024bbe108b77b166b6c0bb5ddb088103ccb8e2775ad92139" exitCode=0 Jul 08 09:12:40.545982 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:40.545569 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" event={"ID":"6df5644a-f34e-4814-af4e-8598341baf1f","Type":"ContainerDied","Data":"d9f4711c1f467656024bbe108b77b166b6c0bb5ddb088103ccb8e2775ad92139"} Jul 08 09:12:43.557955 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:43.557918 2572 generic.go:356] "Generic (PLEG): container finished" podID="6df5644a-f34e-4814-af4e-8598341baf1f" containerID="26e1cbb56d38787b9c2dbe6013b0fccbfee3bae19ebf45892cd4294ae7ac5298" exitCode=0 Jul 08 09:12:43.558357 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:43.557975 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" event={"ID":"6df5644a-f34e-4814-af4e-8598341baf1f","Type":"ContainerDied","Data":"26e1cbb56d38787b9c2dbe6013b0fccbfee3bae19ebf45892cd4294ae7ac5298"} Jul 08 09:12:52.590286 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:52.590246 2572 generic.go:356] "Generic (PLEG): container finished" podID="6df5644a-f34e-4814-af4e-8598341baf1f" containerID="1c9717b986d1be3860a1ee2491772d3ba160d27f8a25b9c53f5c4169528dce95" exitCode=0 Jul 08 09:12:52.590704 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:52.590310 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" event={"ID":"6df5644a-f34e-4814-af4e-8598341baf1f","Type":"ContainerDied","Data":"1c9717b986d1be3860a1ee2491772d3ba160d27f8a25b9c53f5c4169528dce95"} Jul 08 09:12:53.717022 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:53.716996 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:53.811616 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:53.811581 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8k7v5\" (UniqueName: \"kubernetes.io/projected/6df5644a-f34e-4814-af4e-8598341baf1f-kube-api-access-8k7v5\") pod \"6df5644a-f34e-4814-af4e-8598341baf1f\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " Jul 08 09:12:53.811616 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:53.811619 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-bundle\") pod \"6df5644a-f34e-4814-af4e-8598341baf1f\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " Jul 08 09:12:53.811812 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:53.811645 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-util\") pod \"6df5644a-f34e-4814-af4e-8598341baf1f\" (UID: \"6df5644a-f34e-4814-af4e-8598341baf1f\") " Jul 08 09:12:53.812458 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:53.812429 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-bundle" (OuterVolumeSpecName: "bundle") pod "6df5644a-f34e-4814-af4e-8598341baf1f" (UID: "6df5644a-f34e-4814-af4e-8598341baf1f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:12:53.813801 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:53.813779 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6df5644a-f34e-4814-af4e-8598341baf1f-kube-api-access-8k7v5" (OuterVolumeSpecName: "kube-api-access-8k7v5") pod "6df5644a-f34e-4814-af4e-8598341baf1f" (UID: "6df5644a-f34e-4814-af4e-8598341baf1f"). InnerVolumeSpecName "kube-api-access-8k7v5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:12:53.816570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:53.816542 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-util" (OuterVolumeSpecName: "util") pod "6df5644a-f34e-4814-af4e-8598341baf1f" (UID: "6df5644a-f34e-4814-af4e-8598341baf1f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:12:53.912469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:53.912366 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k7v5\" (UniqueName: \"kubernetes.io/projected/6df5644a-f34e-4814-af4e-8598341baf1f-kube-api-access-8k7v5\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:12:53.912469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:53.912408 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-bundle\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:12:53.912469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:53.912421 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df5644a-f34e-4814-af4e-8598341baf1f-util\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:12:54.597838 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:54.597720 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" event={"ID":"6df5644a-f34e-4814-af4e-8598341baf1f","Type":"ContainerDied","Data":"06e9cbf4d87211c7ce68510c8283fa3f19e1ad28332b4a4db99466b4dfdcb64f"} Jul 08 09:12:54.597838 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:54.597780 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5ng9" Jul 08 09:12:54.597838 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:12:54.597828 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06e9cbf4d87211c7ce68510c8283fa3f19e1ad28332b4a4db99466b4dfdcb64f" Jul 08 09:13:06.593574 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.593541 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t"] Jul 08 09:13:06.594035 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.593800 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df5644a-f34e-4814-af4e-8598341baf1f" containerName="util" Jul 08 09:13:06.594035 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.593814 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df5644a-f34e-4814-af4e-8598341baf1f" containerName="util" Jul 08 09:13:06.594035 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.593825 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df5644a-f34e-4814-af4e-8598341baf1f" containerName="pull" Jul 08 09:13:06.594035 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.593833 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df5644a-f34e-4814-af4e-8598341baf1f" containerName="pull" Jul 08 09:13:06.594035 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.593850 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df5644a-f34e-4814-af4e-8598341baf1f" containerName="extract" Jul 08 09:13:06.594035 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.593856 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df5644a-f34e-4814-af4e-8598341baf1f" containerName="extract" Jul 08 09:13:06.594035 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.593905 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6df5644a-f34e-4814-af4e-8598341baf1f" containerName="extract" Jul 08 09:13:06.595736 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.595719 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:06.599315 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.599273 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 09:13:06.600312 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.600288 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 09:13:06.600312 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.600287 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-c5lh2\"" Jul 08 09:13:06.624319 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.624299 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t"] Jul 08 09:13:06.705550 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.705517 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzkt8\" (UniqueName: \"kubernetes.io/projected/bb79187b-2241-486e-bd65-f134b0594cf9-kube-api-access-lzkt8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:06.705715 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.705571 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:06.705715 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.705634 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:06.806835 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.806796 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzkt8\" (UniqueName: \"kubernetes.io/projected/bb79187b-2241-486e-bd65-f134b0594cf9-kube-api-access-lzkt8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:06.806996 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.806845 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:06.806996 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.806880 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:06.807206 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.807191 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:06.807241 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.807223 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:06.820561 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.820536 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzkt8\" (UniqueName: \"kubernetes.io/projected/bb79187b-2241-486e-bd65-f134b0594cf9-kube-api-access-lzkt8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:06.904795 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:06.904673 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:07.092873 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:07.092843 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t"] Jul 08 09:13:07.096083 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:13:07.096042 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb79187b_2241_486e_bd65_f134b0594cf9.slice/crio-c28fcc1dcafed241af0c80fd7e958e5b85f6eeffac8d40cfa6369f34f07c5cc3 WatchSource:0}: Error finding container c28fcc1dcafed241af0c80fd7e958e5b85f6eeffac8d40cfa6369f34f07c5cc3: Status 404 returned error can't find the container with id c28fcc1dcafed241af0c80fd7e958e5b85f6eeffac8d40cfa6369f34f07c5cc3 Jul 08 09:13:07.640157 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:07.640127 2572 generic.go:356] "Generic (PLEG): container finished" podID="bb79187b-2241-486e-bd65-f134b0594cf9" containerID="ab8e5cacddfe607a1489287c4b4921074e29a8d02c5d5afc2ffae3de6c328c30" exitCode=0 Jul 08 09:13:07.640535 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:07.640197 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" event={"ID":"bb79187b-2241-486e-bd65-f134b0594cf9","Type":"ContainerDied","Data":"ab8e5cacddfe607a1489287c4b4921074e29a8d02c5d5afc2ffae3de6c328c30"} Jul 08 09:13:07.640535 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:07.640237 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" event={"ID":"bb79187b-2241-486e-bd65-f134b0594cf9","Type":"ContainerStarted","Data":"c28fcc1dcafed241af0c80fd7e958e5b85f6eeffac8d40cfa6369f34f07c5cc3"} Jul 08 09:13:09.649059 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:09.649024 2572 generic.go:356] "Generic (PLEG): container finished" podID="bb79187b-2241-486e-bd65-f134b0594cf9" containerID="50aff1fb282de463b5023051be6288727c862685a31d04c28576bdae34a70994" exitCode=0 Jul 08 09:13:09.649430 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:09.649111 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" event={"ID":"bb79187b-2241-486e-bd65-f134b0594cf9","Type":"ContainerDied","Data":"50aff1fb282de463b5023051be6288727c862685a31d04c28576bdae34a70994"} Jul 08 09:13:10.654018 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:10.653983 2572 generic.go:356] "Generic (PLEG): container finished" podID="bb79187b-2241-486e-bd65-f134b0594cf9" containerID="c11843ad0335f0152edbfd8480c40f229a76836e507e6ef2bfd764cf7d44d052" exitCode=0 Jul 08 09:13:10.654408 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:10.654054 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" event={"ID":"bb79187b-2241-486e-bd65-f134b0594cf9","Type":"ContainerDied","Data":"c11843ad0335f0152edbfd8480c40f229a76836e507e6ef2bfd764cf7d44d052"} Jul 08 09:13:11.778359 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:11.778330 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:11.845936 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:11.845904 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-util\") pod \"bb79187b-2241-486e-bd65-f134b0594cf9\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " Jul 08 09:13:11.846138 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:11.845977 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-bundle\") pod \"bb79187b-2241-486e-bd65-f134b0594cf9\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " Jul 08 09:13:11.846138 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:11.846001 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzkt8\" (UniqueName: \"kubernetes.io/projected/bb79187b-2241-486e-bd65-f134b0594cf9-kube-api-access-lzkt8\") pod \"bb79187b-2241-486e-bd65-f134b0594cf9\" (UID: \"bb79187b-2241-486e-bd65-f134b0594cf9\") " Jul 08 09:13:11.846383 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:11.846358 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-bundle" (OuterVolumeSpecName: "bundle") pod "bb79187b-2241-486e-bd65-f134b0594cf9" (UID: "bb79187b-2241-486e-bd65-f134b0594cf9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:13:11.848116 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:11.848091 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb79187b-2241-486e-bd65-f134b0594cf9-kube-api-access-lzkt8" (OuterVolumeSpecName: "kube-api-access-lzkt8") pod "bb79187b-2241-486e-bd65-f134b0594cf9" (UID: "bb79187b-2241-486e-bd65-f134b0594cf9"). InnerVolumeSpecName "kube-api-access-lzkt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:13:11.851959 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:11.851937 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-util" (OuterVolumeSpecName: "util") pod "bb79187b-2241-486e-bd65-f134b0594cf9" (UID: "bb79187b-2241-486e-bd65-f134b0594cf9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:13:11.946803 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:11.946692 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-util\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:13:11.946803 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:11.946727 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb79187b-2241-486e-bd65-f134b0594cf9-bundle\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:13:11.946803 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:11.946736 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzkt8\" (UniqueName: \"kubernetes.io/projected/bb79187b-2241-486e-bd65-f134b0594cf9-kube-api-access-lzkt8\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:13:12.414019 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.413987 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nzsmv"] Jul 08 09:13:12.414218 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.414207 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb79187b-2241-486e-bd65-f134b0594cf9" containerName="extract" Jul 08 09:13:12.414264 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.414220 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb79187b-2241-486e-bd65-f134b0594cf9" containerName="extract" Jul 08 09:13:12.414264 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.414237 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb79187b-2241-486e-bd65-f134b0594cf9" containerName="pull" Jul 08 09:13:12.414264 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.414243 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb79187b-2241-486e-bd65-f134b0594cf9" containerName="pull" Jul 08 09:13:12.414264 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.414252 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb79187b-2241-486e-bd65-f134b0594cf9" containerName="util" Jul 08 09:13:12.414264 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.414257 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb79187b-2241-486e-bd65-f134b0594cf9" containerName="util" Jul 08 09:13:12.414416 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.414296 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb79187b-2241-486e-bd65-f134b0594cf9" containerName="extract" Jul 08 09:13:12.416104 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.416089 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" Jul 08 09:13:12.418976 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.418952 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 09:13:12.420058 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.420040 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qmrv5\"" Jul 08 09:13:12.420143 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.420055 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 09:13:12.429372 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.429349 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nzsmv"] Jul 08 09:13:12.551309 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.551258 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdzj8\" (UniqueName: \"kubernetes.io/projected/ac81ac88-97d6-40c6-9d6b-c218843f6866-kube-api-access-vdzj8\") pod \"cert-manager-webhook-6486ffd997-nzsmv\" (UID: \"ac81ac88-97d6-40c6-9d6b-c218843f6866\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" Jul 08 09:13:12.551309 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.551320 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac81ac88-97d6-40c6-9d6b-c218843f6866-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nzsmv\" (UID: \"ac81ac88-97d6-40c6-9d6b-c218843f6866\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" Jul 08 09:13:12.652363 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.652279 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdzj8\" (UniqueName: \"kubernetes.io/projected/ac81ac88-97d6-40c6-9d6b-c218843f6866-kube-api-access-vdzj8\") pod \"cert-manager-webhook-6486ffd997-nzsmv\" (UID: \"ac81ac88-97d6-40c6-9d6b-c218843f6866\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" Jul 08 09:13:12.652363 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.652316 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac81ac88-97d6-40c6-9d6b-c218843f6866-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nzsmv\" (UID: \"ac81ac88-97d6-40c6-9d6b-c218843f6866\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" Jul 08 09:13:12.662832 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.662781 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdzj8\" (UniqueName: \"kubernetes.io/projected/ac81ac88-97d6-40c6-9d6b-c218843f6866-kube-api-access-vdzj8\") pod \"cert-manager-webhook-6486ffd997-nzsmv\" (UID: \"ac81ac88-97d6-40c6-9d6b-c218843f6866\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" Jul 08 09:13:12.663131 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.663103 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" event={"ID":"bb79187b-2241-486e-bd65-f134b0594cf9","Type":"ContainerDied","Data":"c28fcc1dcafed241af0c80fd7e958e5b85f6eeffac8d40cfa6369f34f07c5cc3"} Jul 08 09:13:12.663181 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.663155 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c28fcc1dcafed241af0c80fd7e958e5b85f6eeffac8d40cfa6369f34f07c5cc3" Jul 08 09:13:12.663181 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.663124 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw8f9t" Jul 08 09:13:12.664622 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.664604 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac81ac88-97d6-40c6-9d6b-c218843f6866-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nzsmv\" (UID: \"ac81ac88-97d6-40c6-9d6b-c218843f6866\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" Jul 08 09:13:12.724875 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.724847 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" Jul 08 09:13:12.857896 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:12.857867 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nzsmv"] Jul 08 09:13:12.861191 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:13:12.861152 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac81ac88_97d6_40c6_9d6b_c218843f6866.slice/crio-8b3fb90e7379c98b693705d559954c30f374d9ed96203ce0705e4b03ac4b1367 WatchSource:0}: Error finding container 8b3fb90e7379c98b693705d559954c30f374d9ed96203ce0705e4b03ac4b1367: Status 404 returned error can't find the container with id 8b3fb90e7379c98b693705d559954c30f374d9ed96203ce0705e4b03ac4b1367 Jul 08 09:13:13.667870 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:13.667836 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" event={"ID":"ac81ac88-97d6-40c6-9d6b-c218843f6866","Type":"ContainerStarted","Data":"8b3fb90e7379c98b693705d559954c30f374d9ed96203ce0705e4b03ac4b1367"} Jul 08 09:13:15.681831 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:15.681787 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" event={"ID":"ac81ac88-97d6-40c6-9d6b-c218843f6866","Type":"ContainerStarted","Data":"ffe9c11f8423a856d53cd9dcc54e5678aff236d2ac649a1449bd307dfc806e31"} Jul 08 09:13:15.682205 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:15.681983 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" Jul 08 09:13:15.701073 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:15.701028 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" podStartSLOduration=0.983056721 podStartE2EDuration="3.701013494s" totalImagesPullingTime="2.717956773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:13:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:13:15.698982686 +0000 UTC m=+393.948941968" watchObservedRunningTime="2026-07-08 09:13:15.701013494 +0000 UTC m=+393.950972777" Jul 08 09:13:21.688810 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:21.688730 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nzsmv" Jul 08 09:13:24.424248 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.424211 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n"] Jul 08 09:13:24.426468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.426453 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.429776 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.429735 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 09:13:24.430156 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.430135 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 09:13:24.430922 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.430909 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-c5lh2\"" Jul 08 09:13:24.448385 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.448364 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n"] Jul 08 09:13:24.538601 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.538563 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.538800 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.538621 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2944f\" (UniqueName: \"kubernetes.io/projected/07b2da99-a5fa-472e-8d99-b7dff5b162a0-kube-api-access-2944f\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.538800 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.538648 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.639921 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.639889 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.640106 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.639939 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.640106 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.639990 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2944f\" (UniqueName: \"kubernetes.io/projected/07b2da99-a5fa-472e-8d99-b7dff5b162a0-kube-api-access-2944f\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.640365 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.640340 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.640436 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.640379 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.652415 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.652390 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2944f\" (UniqueName: \"kubernetes.io/projected/07b2da99-a5fa-472e-8d99-b7dff5b162a0-kube-api-access-2944f\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.735515 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.735484 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:24.859241 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:24.859094 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n"] Jul 08 09:13:24.861677 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:13:24.861650 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b2da99_a5fa_472e_8d99_b7dff5b162a0.slice/crio-947bd228419e8b24a1b31b416f78d9a19568d8d1478d4366d8dd7d19c33cabbc WatchSource:0}: Error finding container 947bd228419e8b24a1b31b416f78d9a19568d8d1478d4366d8dd7d19c33cabbc: Status 404 returned error can't find the container with id 947bd228419e8b24a1b31b416f78d9a19568d8d1478d4366d8dd7d19c33cabbc Jul 08 09:13:25.715710 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:25.715675 2572 generic.go:356] "Generic (PLEG): container finished" podID="07b2da99-a5fa-472e-8d99-b7dff5b162a0" containerID="827584a72717c18102c72b27938c5e4413874cfb54c6f1de41607f9eca78b184" exitCode=0 Jul 08 09:13:25.716193 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:25.715726 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" event={"ID":"07b2da99-a5fa-472e-8d99-b7dff5b162a0","Type":"ContainerDied","Data":"827584a72717c18102c72b27938c5e4413874cfb54c6f1de41607f9eca78b184"} Jul 08 09:13:25.716193 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:25.715782 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" event={"ID":"07b2da99-a5fa-472e-8d99-b7dff5b162a0","Type":"ContainerStarted","Data":"947bd228419e8b24a1b31b416f78d9a19568d8d1478d4366d8dd7d19c33cabbc"} Jul 08 09:13:26.721455 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:26.721372 2572 generic.go:356] "Generic (PLEG): container finished" podID="07b2da99-a5fa-472e-8d99-b7dff5b162a0" containerID="13d2428609f6f00ab2f3ab9b4e3731b09db253ff12e0b3392f0712398ef3a248" exitCode=0 Jul 08 09:13:26.721824 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:26.721454 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" event={"ID":"07b2da99-a5fa-472e-8d99-b7dff5b162a0","Type":"ContainerDied","Data":"13d2428609f6f00ab2f3ab9b4e3731b09db253ff12e0b3392f0712398ef3a248"} Jul 08 09:13:27.726697 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:27.726664 2572 generic.go:356] "Generic (PLEG): container finished" podID="07b2da99-a5fa-472e-8d99-b7dff5b162a0" containerID="bb69025d85dd373923fababf9e3c5e8e3fde4c2d5bbeb454a77ee5918a7e61e2" exitCode=0 Jul 08 09:13:27.727068 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:27.726720 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" event={"ID":"07b2da99-a5fa-472e-8d99-b7dff5b162a0","Type":"ContainerDied","Data":"bb69025d85dd373923fababf9e3c5e8e3fde4c2d5bbeb454a77ee5918a7e61e2"} Jul 08 09:13:28.853575 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:28.853545 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:28.971864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:28.971836 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-util\") pod \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " Jul 08 09:13:28.972055 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:28.971880 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-bundle\") pod \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " Jul 08 09:13:28.972055 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:28.971927 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2944f\" (UniqueName: \"kubernetes.io/projected/07b2da99-a5fa-472e-8d99-b7dff5b162a0-kube-api-access-2944f\") pod \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\" (UID: \"07b2da99-a5fa-472e-8d99-b7dff5b162a0\") " Jul 08 09:13:28.972678 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:28.972639 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-bundle" (OuterVolumeSpecName: "bundle") pod "07b2da99-a5fa-472e-8d99-b7dff5b162a0" (UID: "07b2da99-a5fa-472e-8d99-b7dff5b162a0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:13:28.974117 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:28.974093 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b2da99-a5fa-472e-8d99-b7dff5b162a0-kube-api-access-2944f" (OuterVolumeSpecName: "kube-api-access-2944f") pod "07b2da99-a5fa-472e-8d99-b7dff5b162a0" (UID: "07b2da99-a5fa-472e-8d99-b7dff5b162a0"). InnerVolumeSpecName "kube-api-access-2944f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:13:28.977309 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:28.977290 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-util" (OuterVolumeSpecName: "util") pod "07b2da99-a5fa-472e-8d99-b7dff5b162a0" (UID: "07b2da99-a5fa-472e-8d99-b7dff5b162a0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:13:29.072445 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:29.072402 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-util\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:13:29.072445 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:29.072437 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07b2da99-a5fa-472e-8d99-b7dff5b162a0-bundle\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:13:29.072445 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:29.072446 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2944f\" (UniqueName: \"kubernetes.io/projected/07b2da99-a5fa-472e-8d99-b7dff5b162a0-kube-api-access-2944f\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:13:29.735430 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:29.735395 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" event={"ID":"07b2da99-a5fa-472e-8d99-b7dff5b162a0","Type":"ContainerDied","Data":"947bd228419e8b24a1b31b416f78d9a19568d8d1478d4366d8dd7d19c33cabbc"} Jul 08 09:13:29.735597 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:29.735448 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="947bd228419e8b24a1b31b416f78d9a19568d8d1478d4366d8dd7d19c33cabbc" Jul 08 09:13:29.735597 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:29.735482 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqp77n" Jul 08 09:13:37.772452 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.772416 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25"] Jul 08 09:13:37.772904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.772669 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b2da99-a5fa-472e-8d99-b7dff5b162a0" containerName="util" Jul 08 09:13:37.772904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.772681 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b2da99-a5fa-472e-8d99-b7dff5b162a0" containerName="util" Jul 08 09:13:37.772904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.772689 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b2da99-a5fa-472e-8d99-b7dff5b162a0" containerName="pull" Jul 08 09:13:37.772904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.772695 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b2da99-a5fa-472e-8d99-b7dff5b162a0" containerName="pull" Jul 08 09:13:37.772904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.772700 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b2da99-a5fa-472e-8d99-b7dff5b162a0" containerName="extract" Jul 08 09:13:37.772904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.772706 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b2da99-a5fa-472e-8d99-b7dff5b162a0" containerName="extract" Jul 08 09:13:37.772904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.772744 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b2da99-a5fa-472e-8d99-b7dff5b162a0" containerName="extract" Jul 08 09:13:37.775272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.775257 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:37.777902 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.777875 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 09:13:37.779602 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.779581 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 09:13:37.779602 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.779588 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 09:13:37.779807 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.779682 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-j975l\"" Jul 08 09:13:37.780502 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.780480 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 09:13:37.822479 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.822445 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25"] Jul 08 09:13:37.942067 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.942038 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-tmp\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:37.942067 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.942075 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:37.942278 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.942113 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb47t\" (UniqueName: \"kubernetes.io/projected/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-kube-api-access-cb47t\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:37.942278 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.942155 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:37.942278 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:37.942173 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-manifests\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.043625 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.043534 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-tmp\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.043625 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.043577 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.043625 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.043605 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb47t\" (UniqueName: \"kubernetes.io/projected/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-kube-api-access-cb47t\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.043928 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.043638 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.043928 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.043655 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-manifests\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.044090 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.044064 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-manifests\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.045896 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.045865 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-tmp\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.046107 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.046089 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.046167 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.046129 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.058817 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.058793 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb47t\" (UniqueName: \"kubernetes.io/projected/2e8c1ce1-b47a-4a04-a128-d7f172173cfd-kube-api-access-cb47t\") pod \"opendatahub-operator-controller-manager-66d6485f98-f7q25\" (UID: \"2e8c1ce1-b47a-4a04-a128-d7f172173cfd\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.085810 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.085786 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:38.254567 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.254543 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25"] Jul 08 09:13:38.255683 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:13:38.255654 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8c1ce1_b47a_4a04_a128_d7f172173cfd.slice/crio-4b63ebc43837261819d977771453ec7491443c2568174c1ff538891383254d05 WatchSource:0}: Error finding container 4b63ebc43837261819d977771453ec7491443c2568174c1ff538891383254d05: Status 404 returned error can't find the container with id 4b63ebc43837261819d977771453ec7491443c2568174c1ff538891383254d05 Jul 08 09:13:38.766570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:38.766538 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" event={"ID":"2e8c1ce1-b47a-4a04-a128-d7f172173cfd","Type":"ContainerStarted","Data":"4b63ebc43837261819d977771453ec7491443c2568174c1ff538891383254d05"} Jul 08 09:13:41.785402 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:41.785368 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" event={"ID":"2e8c1ce1-b47a-4a04-a128-d7f172173cfd","Type":"ContainerStarted","Data":"1b6315a14b85d76aaaf0a33494d4c4a187912e69452039c6f9f97e3c37626923"} Jul 08 09:13:42.790818 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:42.790780 2572 generic.go:356] "Generic (PLEG): container finished" podID="2e8c1ce1-b47a-4a04-a128-d7f172173cfd" containerID="1b6315a14b85d76aaaf0a33494d4c4a187912e69452039c6f9f97e3c37626923" exitCode=0 Jul 08 09:13:42.791248 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:42.790845 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" event={"ID":"2e8c1ce1-b47a-4a04-a128-d7f172173cfd","Type":"ContainerDied","Data":"1b6315a14b85d76aaaf0a33494d4c4a187912e69452039c6f9f97e3c37626923"} Jul 08 09:13:43.797237 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:43.797195 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" event={"ID":"2e8c1ce1-b47a-4a04-a128-d7f172173cfd","Type":"ContainerStarted","Data":"183715f7dd1cc97442bf7f759d5b277189d574551ee9892b34ab671e08d580c8"} Jul 08 09:13:43.797631 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:43.797284 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:54.804206 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:54.804173 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" Jul 08 09:13:54.829941 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:54.829884 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-f7q25" podStartSLOduration=13.097440689 podStartE2EDuration="17.829862325s" totalImagesPullingTime="3.725540869s" totalInitContainerRuntime="1.006880767s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:13:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:13:43.857080276 +0000 UTC m=+422.107039560" watchObservedRunningTime="2026-07-08 09:13:54.829862325 +0000 UTC m=+433.079821608" Jul 08 09:13:57.666982 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.666947 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77"] Jul 08 09:13:57.674982 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.674959 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.680524 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.680503 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 09:13:57.682036 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.682021 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 09:13:57.682546 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.682532 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 09:13:57.682650 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.682544 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vdj5w\"" Jul 08 09:13:57.682650 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.682550 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 09:13:57.683057 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.683042 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 09:13:57.698836 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.698813 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77"] Jul 08 09:13:57.794677 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.794640 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f68bp\" (UniqueName: \"kubernetes.io/projected/f90c3f88-744e-43f7-9727-a80bffe90bc3-kube-api-access-f68bp\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.794887 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.794692 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f90c3f88-744e-43f7-9727-a80bffe90bc3-metrics-cert\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.794887 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.794783 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f90c3f88-744e-43f7-9727-a80bffe90bc3-cert\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.794887 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.794814 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f90c3f88-744e-43f7-9727-a80bffe90bc3-manager-config\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.895849 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.895807 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f90c3f88-744e-43f7-9727-a80bffe90bc3-manager-config\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.896052 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.895972 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f68bp\" (UniqueName: \"kubernetes.io/projected/f90c3f88-744e-43f7-9727-a80bffe90bc3-kube-api-access-f68bp\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.896052 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.896003 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f90c3f88-744e-43f7-9727-a80bffe90bc3-metrics-cert\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.896052 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.896038 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f90c3f88-744e-43f7-9727-a80bffe90bc3-cert\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.896565 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.896539 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f90c3f88-744e-43f7-9727-a80bffe90bc3-manager-config\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.898531 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.898510 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f90c3f88-744e-43f7-9727-a80bffe90bc3-cert\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.898664 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.898643 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f90c3f88-744e-43f7-9727-a80bffe90bc3-metrics-cert\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.943741 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.943668 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f68bp\" (UniqueName: \"kubernetes.io/projected/f90c3f88-744e-43f7-9727-a80bffe90bc3-kube-api-access-f68bp\") pod \"lws-controller-manager-6d9788fbb6-djv77\" (UID: \"f90c3f88-744e-43f7-9727-a80bffe90bc3\") " pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:57.983813 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:57.983771 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:13:58.149929 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:58.149898 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77"] Jul 08 09:13:58.151328 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:13:58.151293 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf90c3f88_744e_43f7_9727_a80bffe90bc3.slice/crio-6dd68f62e79369b8e3c3fd3e8e5d8bc77035a425e1d5f977cb4d893517189f16 WatchSource:0}: Error finding container 6dd68f62e79369b8e3c3fd3e8e5d8bc77035a425e1d5f977cb4d893517189f16: Status 404 returned error can't find the container with id 6dd68f62e79369b8e3c3fd3e8e5d8bc77035a425e1d5f977cb4d893517189f16 Jul 08 09:13:58.857873 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:13:58.857835 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" event={"ID":"f90c3f88-744e-43f7-9727-a80bffe90bc3","Type":"ContainerStarted","Data":"6dd68f62e79369b8e3c3fd3e8e5d8bc77035a425e1d5f977cb4d893517189f16"} Jul 08 09:14:00.868555 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:00.868524 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" event={"ID":"f90c3f88-744e-43f7-9727-a80bffe90bc3","Type":"ContainerStarted","Data":"e763628156b14d14cd988ef22422493b1f99de5b9a10ce539b2fb8b6b5a9ce9c"} Jul 08 09:14:00.868981 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:00.868659 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:14:00.891246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:00.891183 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" podStartSLOduration=2.064587037 podStartE2EDuration="3.89116052s" totalImagesPullingTime="1.826573483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:14:00.8896095 +0000 UTC m=+439.139568784" watchObservedRunningTime="2026-07-08 09:14:00.89116052 +0000 UTC m=+439.141119804" Jul 08 09:14:05.095306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.095232 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48"] Jul 08 09:14:05.098458 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.098441 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.101493 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.101463 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 09:14:05.101692 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.101671 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-w4rnw\"" Jul 08 09:14:05.102060 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.102041 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 09:14:05.102126 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.102097 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 09:14:05.102832 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.102815 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 09:14:05.120448 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.120420 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48"] Jul 08 09:14:05.152215 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.152181 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/157ed47a-6e4e-4c64-8708-e119ddfb0575-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.152215 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.152215 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/157ed47a-6e4e-4c64-8708-e119ddfb0575-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.152417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.152232 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.152417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.152272 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.152417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.152343 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m2p2\" (UniqueName: \"kubernetes.io/projected/157ed47a-6e4e-4c64-8708-e119ddfb0575-kube-api-access-7m2p2\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.152417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.152384 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.152417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.152406 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.253442 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.253402 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.253442 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.253449 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.253650 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.253494 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/157ed47a-6e4e-4c64-8708-e119ddfb0575-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.253650 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.253532 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/157ed47a-6e4e-4c64-8708-e119ddfb0575-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.253650 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.253557 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.253650 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.253585 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.253847 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.253668 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m2p2\" (UniqueName: \"kubernetes.io/projected/157ed47a-6e4e-4c64-8708-e119ddfb0575-kube-api-access-7m2p2\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.254382 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.254349 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.256106 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.256078 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/157ed47a-6e4e-4c64-8708-e119ddfb0575-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.256214 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.256192 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.256255 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.256206 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/157ed47a-6e4e-4c64-8708-e119ddfb0575-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.256255 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.256197 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.265642 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.265616 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/157ed47a-6e4e-4c64-8708-e119ddfb0575-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.281951 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.281921 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m2p2\" (UniqueName: \"kubernetes.io/projected/157ed47a-6e4e-4c64-8708-e119ddfb0575-kube-api-access-7m2p2\") pod \"istiod-openshift-gateway-94bb8fbfd-c6h48\" (UID: \"157ed47a-6e4e-4c64-8708-e119ddfb0575\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.408149 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.408063 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:05.545716 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.545690 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48"] Jul 08 09:14:05.547704 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:14:05.547679 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod157ed47a_6e4e_4c64_8708_e119ddfb0575.slice/crio-6518bf47e864e877e7758b31447a3c9968a340b5a747c16e9c4b9053fa29bb0d WatchSource:0}: Error finding container 6518bf47e864e877e7758b31447a3c9968a340b5a747c16e9c4b9053fa29bb0d: Status 404 returned error can't find the container with id 6518bf47e864e877e7758b31447a3c9968a340b5a747c16e9c4b9053fa29bb0d Jul 08 09:14:05.888199 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:05.888161 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" event={"ID":"157ed47a-6e4e-4c64-8708-e119ddfb0575","Type":"ContainerStarted","Data":"6518bf47e864e877e7758b31447a3c9968a340b5a747c16e9c4b9053fa29bb0d"} Jul 08 09:14:08.036512 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:08.036475 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:14:08.036824 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:08.036544 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:14:08.911631 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:08.911593 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" event={"ID":"157ed47a-6e4e-4c64-8708-e119ddfb0575","Type":"ContainerStarted","Data":"d5c11a4c2f1895541de6e8a3dcb5f0097ec1d987e0e5fae7f5340236c094a702"} Jul 08 09:14:08.911906 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:08.911686 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:08.959948 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:08.959898 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" podStartSLOduration=1.473266629 podStartE2EDuration="3.959883028s" totalImagesPullingTime="2.486616399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:14:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:14:08.958007788 +0000 UTC m=+447.207967083" watchObservedRunningTime="2026-07-08 09:14:08.959883028 +0000 UTC m=+447.209842310" Jul 08 09:14:09.917455 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:09.917430 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c6h48" Jul 08 09:14:11.874453 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:14:11.874423 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d9788fbb6-djv77" Jul 08 09:15:13.965320 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:13.965282 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64"] Jul 08 09:15:13.968726 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:13.968706 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:13.971527 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:13.971501 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 09:15:13.971527 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:13.971510 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 09:15:13.972531 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:13.972518 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rxqb9\"" Jul 08 09:15:13.976502 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:13.976477 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64"] Jul 08 09:15:14.079889 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.079845 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:14.080082 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.079916 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8brc\" (UniqueName: \"kubernetes.io/projected/973fd9b5-2707-43ec-81b7-3db96c8385a0-kube-api-access-m8brc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:14.080082 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.079961 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:14.180891 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.180859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:14.181090 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.180924 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:14.181090 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.180978 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8brc\" (UniqueName: \"kubernetes.io/projected/973fd9b5-2707-43ec-81b7-3db96c8385a0-kube-api-access-m8brc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:14.181342 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.181321 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:14.181419 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.181365 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:14.190182 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.190144 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8brc\" (UniqueName: \"kubernetes.io/projected/973fd9b5-2707-43ec-81b7-3db96c8385a0-kube-api-access-m8brc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:14.279131 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.279095 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:14.410337 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.410312 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64"] Jul 08 09:15:14.412735 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:15:14.412702 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod973fd9b5_2707_43ec_81b7_3db96c8385a0.slice/crio-be9a2be19c62efc1b6af0f9a09638cf75d102e38bbe50e8b38735b7483bc85e9 WatchSource:0}: Error finding container be9a2be19c62efc1b6af0f9a09638cf75d102e38bbe50e8b38735b7483bc85e9: Status 404 returned error can't find the container with id be9a2be19c62efc1b6af0f9a09638cf75d102e38bbe50e8b38735b7483bc85e9 Jul 08 09:15:14.563792 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.563688 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88"] Jul 08 09:15:14.566928 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.566912 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.578510 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.578478 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88"] Jul 08 09:15:14.686380 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.686341 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9vjn\" (UniqueName: \"kubernetes.io/projected/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-kube-api-access-h9vjn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.686380 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.686384 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.686584 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.686440 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.787208 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.787173 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9vjn\" (UniqueName: \"kubernetes.io/projected/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-kube-api-access-h9vjn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.787208 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.787213 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.787469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.787244 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.787590 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.787571 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.787650 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.787626 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.796387 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.796361 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9vjn\" (UniqueName: \"kubernetes.io/projected/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-kube-api-access-h9vjn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.876331 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.876236 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:14.987239 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.987206 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch"] Jul 08 09:15:14.990822 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:14.990801 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.000895 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.000866 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch"] Jul 08 09:15:15.004367 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.004338 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88"] Jul 08 09:15:15.008221 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:15:15.008194 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97af5b9f_a944_4d3f_9547_9c9404eaeb3d.slice/crio-4ddeadab2d4d033390a5105bc970d79d25ba3355f6e42f3c9892d17858b16168 WatchSource:0}: Error finding container 4ddeadab2d4d033390a5105bc970d79d25ba3355f6e42f3c9892d17858b16168: Status 404 returned error can't find the container with id 4ddeadab2d4d033390a5105bc970d79d25ba3355f6e42f3c9892d17858b16168 Jul 08 09:15:15.090964 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.090929 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxz9m\" (UniqueName: \"kubernetes.io/projected/7ceef47b-7a79-4a93-b828-ef7082f35650-kube-api-access-hxz9m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.091192 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.090976 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.091192 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.091053 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.173766 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.173661 2572 generic.go:356] "Generic (PLEG): container finished" podID="973fd9b5-2707-43ec-81b7-3db96c8385a0" containerID="535c2da23da2a8f9a0792d412125e565f8272b7db647ac5317128fb0a3bebe44" exitCode=0 Jul 08 09:15:15.173938 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.173779 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" event={"ID":"973fd9b5-2707-43ec-81b7-3db96c8385a0","Type":"ContainerDied","Data":"535c2da23da2a8f9a0792d412125e565f8272b7db647ac5317128fb0a3bebe44"} Jul 08 09:15:15.173938 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.173825 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" event={"ID":"973fd9b5-2707-43ec-81b7-3db96c8385a0","Type":"ContainerStarted","Data":"be9a2be19c62efc1b6af0f9a09638cf75d102e38bbe50e8b38735b7483bc85e9"} Jul 08 09:15:15.175311 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.175284 2572 generic.go:356] "Generic (PLEG): container finished" podID="97af5b9f-a944-4d3f-9547-9c9404eaeb3d" containerID="b83c705aa69735b45142a05a8be1aac15a9e50f4e61e6b0060d65eaad449e8e8" exitCode=0 Jul 08 09:15:15.175409 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.175349 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" event={"ID":"97af5b9f-a944-4d3f-9547-9c9404eaeb3d","Type":"ContainerDied","Data":"b83c705aa69735b45142a05a8be1aac15a9e50f4e61e6b0060d65eaad449e8e8"} Jul 08 09:15:15.175409 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.175381 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" event={"ID":"97af5b9f-a944-4d3f-9547-9c9404eaeb3d","Type":"ContainerStarted","Data":"4ddeadab2d4d033390a5105bc970d79d25ba3355f6e42f3c9892d17858b16168"} Jul 08 09:15:15.192188 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.192118 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.192188 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.192190 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxz9m\" (UniqueName: \"kubernetes.io/projected/7ceef47b-7a79-4a93-b828-ef7082f35650-kube-api-access-hxz9m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.192489 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.192234 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.192681 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.192627 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.193183 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.193159 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.202220 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.202189 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxz9m\" (UniqueName: \"kubernetes.io/projected/7ceef47b-7a79-4a93-b828-ef7082f35650-kube-api-access-hxz9m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.300582 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.300546 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:15.368910 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.368880 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg"] Jul 08 09:15:15.372382 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.372364 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.381255 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.381218 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg"] Jul 08 09:15:15.435174 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.435096 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch"] Jul 08 09:15:15.437899 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:15:15.437872 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ceef47b_7a79_4a93_b828_ef7082f35650.slice/crio-b4aba664f82fb8325151c843aa1ae9ad79ee7d525ff4b751b6719bb172d471a0 WatchSource:0}: Error finding container b4aba664f82fb8325151c843aa1ae9ad79ee7d525ff4b751b6719bb172d471a0: Status 404 returned error can't find the container with id b4aba664f82fb8325151c843aa1ae9ad79ee7d525ff4b751b6719bb172d471a0 Jul 08 09:15:15.494821 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.494801 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.494902 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.494845 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p77sc\" (UniqueName: \"kubernetes.io/projected/5cb68d01-0bec-48b1-91ab-145355aad160-kube-api-access-p77sc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.494954 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.494913 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.595348 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.595308 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p77sc\" (UniqueName: \"kubernetes.io/projected/5cb68d01-0bec-48b1-91ab-145355aad160-kube-api-access-p77sc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.595546 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.595367 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.595546 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.595414 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.595785 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.595739 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.595855 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.595821 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.606552 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.606525 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p77sc\" (UniqueName: \"kubernetes.io/projected/5cb68d01-0bec-48b1-91ab-145355aad160-kube-api-access-p77sc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.683511 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.683473 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:15.810910 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:15.810838 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg"] Jul 08 09:15:15.814931 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:15:15.814894 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb68d01_0bec_48b1_91ab_145355aad160.slice/crio-ce39e3835857f5c7a9aaad16f1766d63db5b9f428f940d8b2a5ab48efc5954de WatchSource:0}: Error finding container ce39e3835857f5c7a9aaad16f1766d63db5b9f428f940d8b2a5ab48efc5954de: Status 404 returned error can't find the container with id ce39e3835857f5c7a9aaad16f1766d63db5b9f428f940d8b2a5ab48efc5954de Jul 08 09:15:16.183464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:16.183430 2572 generic.go:356] "Generic (PLEG): container finished" podID="7ceef47b-7a79-4a93-b828-ef7082f35650" containerID="baac9c3a867ddf044557ec4208004ce45ec4052441018ad6004ecce9fd806eab" exitCode=0 Jul 08 09:15:16.183891 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:16.183479 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" event={"ID":"7ceef47b-7a79-4a93-b828-ef7082f35650","Type":"ContainerDied","Data":"baac9c3a867ddf044557ec4208004ce45ec4052441018ad6004ecce9fd806eab"} Jul 08 09:15:16.183891 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:16.183533 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" event={"ID":"7ceef47b-7a79-4a93-b828-ef7082f35650","Type":"ContainerStarted","Data":"b4aba664f82fb8325151c843aa1ae9ad79ee7d525ff4b751b6719bb172d471a0"} Jul 08 09:15:16.185460 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:16.185440 2572 generic.go:356] "Generic (PLEG): container finished" podID="97af5b9f-a944-4d3f-9547-9c9404eaeb3d" containerID="67e73b84424673f8040ab422d34d8fcd149981645cfec9b70dfaf5a3b573c777" exitCode=0 Jul 08 09:15:16.185596 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:16.185502 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" event={"ID":"97af5b9f-a944-4d3f-9547-9c9404eaeb3d","Type":"ContainerDied","Data":"67e73b84424673f8040ab422d34d8fcd149981645cfec9b70dfaf5a3b573c777"} Jul 08 09:15:16.186974 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:16.186953 2572 generic.go:356] "Generic (PLEG): container finished" podID="5cb68d01-0bec-48b1-91ab-145355aad160" containerID="3e05493c975a89a633dac6ca70125e247e635c8eafea94d6906434c65ea1fb2c" exitCode=0 Jul 08 09:15:16.187076 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:16.186995 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" event={"ID":"5cb68d01-0bec-48b1-91ab-145355aad160","Type":"ContainerDied","Data":"3e05493c975a89a633dac6ca70125e247e635c8eafea94d6906434c65ea1fb2c"} Jul 08 09:15:16.187076 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:16.187021 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" event={"ID":"5cb68d01-0bec-48b1-91ab-145355aad160","Type":"ContainerStarted","Data":"ce39e3835857f5c7a9aaad16f1766d63db5b9f428f940d8b2a5ab48efc5954de"} Jul 08 09:15:17.193331 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:17.193295 2572 generic.go:356] "Generic (PLEG): container finished" podID="5cb68d01-0bec-48b1-91ab-145355aad160" containerID="713408c5c57eabbd9697d9d990129ed043844efe594026c364c28ccb93aa1a77" exitCode=0 Jul 08 09:15:17.193820 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:17.193383 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" event={"ID":"5cb68d01-0bec-48b1-91ab-145355aad160","Type":"ContainerDied","Data":"713408c5c57eabbd9697d9d990129ed043844efe594026c364c28ccb93aa1a77"} Jul 08 09:15:17.195270 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:17.195250 2572 generic.go:356] "Generic (PLEG): container finished" podID="7ceef47b-7a79-4a93-b828-ef7082f35650" containerID="c3ed2a7c4e008f47a5cfdcd51de7707c08d2e9c8bc0de6ba83e66f297f60e69c" exitCode=0 Jul 08 09:15:17.195363 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:17.195337 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" event={"ID":"7ceef47b-7a79-4a93-b828-ef7082f35650","Type":"ContainerDied","Data":"c3ed2a7c4e008f47a5cfdcd51de7707c08d2e9c8bc0de6ba83e66f297f60e69c"} Jul 08 09:15:17.197296 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:17.197269 2572 generic.go:356] "Generic (PLEG): container finished" podID="973fd9b5-2707-43ec-81b7-3db96c8385a0" containerID="ff11e4f668e3331d21ce318408ac3059699a07d76c0edc8b934c0b51a9718c21" exitCode=0 Jul 08 09:15:17.197411 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:17.197344 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" event={"ID":"973fd9b5-2707-43ec-81b7-3db96c8385a0","Type":"ContainerDied","Data":"ff11e4f668e3331d21ce318408ac3059699a07d76c0edc8b934c0b51a9718c21"} Jul 08 09:15:17.199569 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:17.199549 2572 generic.go:356] "Generic (PLEG): container finished" podID="97af5b9f-a944-4d3f-9547-9c9404eaeb3d" containerID="81c515ff53f7f604eaae9440b2fd119904eb85c6606e617ed5913431deb151ad" exitCode=0 Jul 08 09:15:17.199662 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:17.199575 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" event={"ID":"97af5b9f-a944-4d3f-9547-9c9404eaeb3d","Type":"ContainerDied","Data":"81c515ff53f7f604eaae9440b2fd119904eb85c6606e617ed5913431deb151ad"} Jul 08 09:15:18.205946 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.205913 2572 generic.go:356] "Generic (PLEG): container finished" podID="973fd9b5-2707-43ec-81b7-3db96c8385a0" containerID="98fa82a9a0269e968a2ebbcb491d12759009b6e2df86c2cea575c9f21208897a" exitCode=0 Jul 08 09:15:18.206283 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.205946 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" event={"ID":"973fd9b5-2707-43ec-81b7-3db96c8385a0","Type":"ContainerDied","Data":"98fa82a9a0269e968a2ebbcb491d12759009b6e2df86c2cea575c9f21208897a"} Jul 08 09:15:18.207870 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.207850 2572 generic.go:356] "Generic (PLEG): container finished" podID="5cb68d01-0bec-48b1-91ab-145355aad160" containerID="5e37b5c100995d7246974baca345bf6c3aa19d8835ac0a3ad8eb2429f5d4a2df" exitCode=0 Jul 08 09:15:18.208105 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.207920 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" event={"ID":"5cb68d01-0bec-48b1-91ab-145355aad160","Type":"ContainerDied","Data":"5e37b5c100995d7246974baca345bf6c3aa19d8835ac0a3ad8eb2429f5d4a2df"} Jul 08 09:15:18.209897 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.209876 2572 generic.go:356] "Generic (PLEG): container finished" podID="7ceef47b-7a79-4a93-b828-ef7082f35650" containerID="b1c590f48dac02bc0f9dcb3b7143813c3125931d76d2c6598412ab6b250b9ad3" exitCode=0 Jul 08 09:15:18.209986 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.209930 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" event={"ID":"7ceef47b-7a79-4a93-b828-ef7082f35650","Type":"ContainerDied","Data":"b1c590f48dac02bc0f9dcb3b7143813c3125931d76d2c6598412ab6b250b9ad3"} Jul 08 09:15:18.333330 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.333307 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:18.421933 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.421897 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-util\") pod \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " Jul 08 09:15:18.422130 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.421980 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-bundle\") pod \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " Jul 08 09:15:18.422130 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.422046 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9vjn\" (UniqueName: \"kubernetes.io/projected/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-kube-api-access-h9vjn\") pod \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\" (UID: \"97af5b9f-a944-4d3f-9547-9c9404eaeb3d\") " Jul 08 09:15:18.422717 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.422687 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-bundle" (OuterVolumeSpecName: "bundle") pod "97af5b9f-a944-4d3f-9547-9c9404eaeb3d" (UID: "97af5b9f-a944-4d3f-9547-9c9404eaeb3d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:15:18.424803 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.424781 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-kube-api-access-h9vjn" (OuterVolumeSpecName: "kube-api-access-h9vjn") pod "97af5b9f-a944-4d3f-9547-9c9404eaeb3d" (UID: "97af5b9f-a944-4d3f-9547-9c9404eaeb3d"). InnerVolumeSpecName "kube-api-access-h9vjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:15:18.427861 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.427837 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-util" (OuterVolumeSpecName: "util") pod "97af5b9f-a944-4d3f-9547-9c9404eaeb3d" (UID: "97af5b9f-a944-4d3f-9547-9c9404eaeb3d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:15:18.527983 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.523820 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-bundle\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:18.527983 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.523863 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9vjn\" (UniqueName: \"kubernetes.io/projected/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-kube-api-access-h9vjn\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:18.527983 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:18.523879 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97af5b9f-a944-4d3f-9547-9c9404eaeb3d-util\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:19.216592 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.216557 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" Jul 08 09:15:19.216592 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.216581 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88" event={"ID":"97af5b9f-a944-4d3f-9547-9c9404eaeb3d","Type":"ContainerDied","Data":"4ddeadab2d4d033390a5105bc970d79d25ba3355f6e42f3c9892d17858b16168"} Jul 08 09:15:19.217215 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.216657 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ddeadab2d4d033390a5105bc970d79d25ba3355f6e42f3c9892d17858b16168" Jul 08 09:15:19.364183 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.364157 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:19.387296 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.387277 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:19.390207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.390186 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:19.534134 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534102 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxz9m\" (UniqueName: \"kubernetes.io/projected/7ceef47b-7a79-4a93-b828-ef7082f35650-kube-api-access-hxz9m\") pod \"7ceef47b-7a79-4a93-b828-ef7082f35650\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " Jul 08 09:15:19.534331 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534144 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-util\") pod \"7ceef47b-7a79-4a93-b828-ef7082f35650\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " Jul 08 09:15:19.534331 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534161 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-util\") pod \"5cb68d01-0bec-48b1-91ab-145355aad160\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " Jul 08 09:15:19.534331 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534186 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-util\") pod \"973fd9b5-2707-43ec-81b7-3db96c8385a0\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " Jul 08 09:15:19.534331 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534208 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-bundle\") pod \"973fd9b5-2707-43ec-81b7-3db96c8385a0\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " Jul 08 09:15:19.534331 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534223 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p77sc\" (UniqueName: \"kubernetes.io/projected/5cb68d01-0bec-48b1-91ab-145355aad160-kube-api-access-p77sc\") pod \"5cb68d01-0bec-48b1-91ab-145355aad160\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " Jul 08 09:15:19.534331 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534282 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-bundle\") pod \"5cb68d01-0bec-48b1-91ab-145355aad160\" (UID: \"5cb68d01-0bec-48b1-91ab-145355aad160\") " Jul 08 09:15:19.534331 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534301 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8brc\" (UniqueName: \"kubernetes.io/projected/973fd9b5-2707-43ec-81b7-3db96c8385a0-kube-api-access-m8brc\") pod \"973fd9b5-2707-43ec-81b7-3db96c8385a0\" (UID: \"973fd9b5-2707-43ec-81b7-3db96c8385a0\") " Jul 08 09:15:19.534786 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534711 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-bundle" (OuterVolumeSpecName: "bundle") pod "973fd9b5-2707-43ec-81b7-3db96c8385a0" (UID: "973fd9b5-2707-43ec-81b7-3db96c8385a0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:15:19.535097 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534909 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-bundle\") pod \"7ceef47b-7a79-4a93-b828-ef7082f35650\" (UID: \"7ceef47b-7a79-4a93-b828-ef7082f35650\") " Jul 08 09:15:19.535097 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.534916 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-bundle" (OuterVolumeSpecName: "bundle") pod "5cb68d01-0bec-48b1-91ab-145355aad160" (UID: "5cb68d01-0bec-48b1-91ab-145355aad160"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:15:19.535493 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.535414 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-bundle\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:19.535493 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.535443 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-bundle\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:19.535673 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.535545 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-bundle" (OuterVolumeSpecName: "bundle") pod "7ceef47b-7a79-4a93-b828-ef7082f35650" (UID: "7ceef47b-7a79-4a93-b828-ef7082f35650"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:15:19.536632 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.536606 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb68d01-0bec-48b1-91ab-145355aad160-kube-api-access-p77sc" (OuterVolumeSpecName: "kube-api-access-p77sc") pod "5cb68d01-0bec-48b1-91ab-145355aad160" (UID: "5cb68d01-0bec-48b1-91ab-145355aad160"). InnerVolumeSpecName "kube-api-access-p77sc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:15:19.536795 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.536776 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ceef47b-7a79-4a93-b828-ef7082f35650-kube-api-access-hxz9m" (OuterVolumeSpecName: "kube-api-access-hxz9m") pod "7ceef47b-7a79-4a93-b828-ef7082f35650" (UID: "7ceef47b-7a79-4a93-b828-ef7082f35650"). InnerVolumeSpecName "kube-api-access-hxz9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:15:19.537128 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.537102 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/973fd9b5-2707-43ec-81b7-3db96c8385a0-kube-api-access-m8brc" (OuterVolumeSpecName: "kube-api-access-m8brc") pod "973fd9b5-2707-43ec-81b7-3db96c8385a0" (UID: "973fd9b5-2707-43ec-81b7-3db96c8385a0"). InnerVolumeSpecName "kube-api-access-m8brc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:15:19.540846 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.540822 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-util" (OuterVolumeSpecName: "util") pod "973fd9b5-2707-43ec-81b7-3db96c8385a0" (UID: "973fd9b5-2707-43ec-81b7-3db96c8385a0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:15:19.540925 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.540905 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-util" (OuterVolumeSpecName: "util") pod "5cb68d01-0bec-48b1-91ab-145355aad160" (UID: "5cb68d01-0bec-48b1-91ab-145355aad160"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:15:19.540987 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.540968 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-util" (OuterVolumeSpecName: "util") pod "7ceef47b-7a79-4a93-b828-ef7082f35650" (UID: "7ceef47b-7a79-4a93-b828-ef7082f35650"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:15:19.636288 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.636246 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8brc\" (UniqueName: \"kubernetes.io/projected/973fd9b5-2707-43ec-81b7-3db96c8385a0-kube-api-access-m8brc\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:19.636288 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.636279 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-bundle\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:19.636288 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.636291 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxz9m\" (UniqueName: \"kubernetes.io/projected/7ceef47b-7a79-4a93-b828-ef7082f35650-kube-api-access-hxz9m\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:19.636288 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.636299 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ceef47b-7a79-4a93-b828-ef7082f35650-util\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:19.636548 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.636307 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cb68d01-0bec-48b1-91ab-145355aad160-util\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:19.636548 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.636315 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/973fd9b5-2707-43ec-81b7-3db96c8385a0-util\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:19.636548 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:19.636322 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p77sc\" (UniqueName: \"kubernetes.io/projected/5cb68d01-0bec-48b1-91ab-145355aad160-kube-api-access-p77sc\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:15:20.222959 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:20.222928 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" Jul 08 09:15:20.222959 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:20.222947 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg" event={"ID":"5cb68d01-0bec-48b1-91ab-145355aad160","Type":"ContainerDied","Data":"ce39e3835857f5c7a9aaad16f1766d63db5b9f428f940d8b2a5ab48efc5954de"} Jul 08 09:15:20.223511 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:20.222992 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce39e3835857f5c7a9aaad16f1766d63db5b9f428f940d8b2a5ab48efc5954de" Jul 08 09:15:20.224807 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:20.224782 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" event={"ID":"7ceef47b-7a79-4a93-b828-ef7082f35650","Type":"ContainerDied","Data":"b4aba664f82fb8325151c843aa1ae9ad79ee7d525ff4b751b6719bb172d471a0"} Jul 08 09:15:20.224934 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:20.224823 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4aba664f82fb8325151c843aa1ae9ad79ee7d525ff4b751b6719bb172d471a0" Jul 08 09:15:20.224934 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:20.224839 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch" Jul 08 09:15:20.226780 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:20.226741 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" event={"ID":"973fd9b5-2707-43ec-81b7-3db96c8385a0","Type":"ContainerDied","Data":"be9a2be19c62efc1b6af0f9a09638cf75d102e38bbe50e8b38735b7483bc85e9"} Jul 08 09:15:20.226882 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:20.226801 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be9a2be19c62efc1b6af0f9a09638cf75d102e38bbe50e8b38735b7483bc85e9" Jul 08 09:15:20.226882 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:20.226764 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64" Jul 08 09:15:58.176786 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.176685 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw"] Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177002 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97af5b9f-a944-4d3f-9547-9c9404eaeb3d" containerName="util" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177016 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="97af5b9f-a944-4d3f-9547-9c9404eaeb3d" containerName="util" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177024 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97af5b9f-a944-4d3f-9547-9c9404eaeb3d" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177030 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="97af5b9f-a944-4d3f-9547-9c9404eaeb3d" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177037 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ceef47b-7a79-4a93-b828-ef7082f35650" containerName="util" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177043 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ceef47b-7a79-4a93-b828-ef7082f35650" containerName="util" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177050 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97af5b9f-a944-4d3f-9547-9c9404eaeb3d" containerName="pull" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177056 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="97af5b9f-a944-4d3f-9547-9c9404eaeb3d" containerName="pull" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177061 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="973fd9b5-2707-43ec-81b7-3db96c8385a0" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177066 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="973fd9b5-2707-43ec-81b7-3db96c8385a0" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177074 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ceef47b-7a79-4a93-b828-ef7082f35650" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177079 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ceef47b-7a79-4a93-b828-ef7082f35650" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177084 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb68d01-0bec-48b1-91ab-145355aad160" containerName="util" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177089 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb68d01-0bec-48b1-91ab-145355aad160" containerName="util" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177096 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="973fd9b5-2707-43ec-81b7-3db96c8385a0" containerName="pull" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177100 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="973fd9b5-2707-43ec-81b7-3db96c8385a0" containerName="pull" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177108 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ceef47b-7a79-4a93-b828-ef7082f35650" containerName="pull" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177112 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ceef47b-7a79-4a93-b828-ef7082f35650" containerName="pull" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177118 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb68d01-0bec-48b1-91ab-145355aad160" containerName="pull" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177123 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb68d01-0bec-48b1-91ab-145355aad160" containerName="pull" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177128 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb68d01-0bec-48b1-91ab-145355aad160" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177133 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb68d01-0bec-48b1-91ab-145355aad160" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177148 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="973fd9b5-2707-43ec-81b7-3db96c8385a0" containerName="util" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177154 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="973fd9b5-2707-43ec-81b7-3db96c8385a0" containerName="util" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177225 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="97af5b9f-a944-4d3f-9547-9c9404eaeb3d" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177232 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb68d01-0bec-48b1-91ab-145355aad160" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177237 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ceef47b-7a79-4a93-b828-ef7082f35650" containerName="extract" Jul 08 09:15:58.177246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.177244 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="973fd9b5-2707-43ec-81b7-3db96c8385a0" containerName="extract" Jul 08 09:15:58.181376 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.181356 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:15:58.184285 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.184263 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 09:15:58.184411 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.184263 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 09:15:58.184455 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.184448 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2tbss\"" Jul 08 09:15:58.196093 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.196067 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw"] Jul 08 09:15:58.245237 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.245206 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50c5d096-071b-432d-888b-8f033d4af207-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hjbfw\" (UID: \"50c5d096-071b-432d-888b-8f033d4af207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:15:58.245237 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.245242 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjxnc\" (UniqueName: \"kubernetes.io/projected/50c5d096-071b-432d-888b-8f033d4af207-kube-api-access-jjxnc\") pod \"kuadrant-operator-controller-manager-55467d9f7-hjbfw\" (UID: \"50c5d096-071b-432d-888b-8f033d4af207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:15:58.345885 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.345848 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50c5d096-071b-432d-888b-8f033d4af207-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hjbfw\" (UID: \"50c5d096-071b-432d-888b-8f033d4af207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:15:58.345885 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.345899 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjxnc\" (UniqueName: \"kubernetes.io/projected/50c5d096-071b-432d-888b-8f033d4af207-kube-api-access-jjxnc\") pod \"kuadrant-operator-controller-manager-55467d9f7-hjbfw\" (UID: \"50c5d096-071b-432d-888b-8f033d4af207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:15:58.346256 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.346231 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50c5d096-071b-432d-888b-8f033d4af207-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hjbfw\" (UID: \"50c5d096-071b-432d-888b-8f033d4af207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:15:58.355655 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.355623 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjxnc\" (UniqueName: \"kubernetes.io/projected/50c5d096-071b-432d-888b-8f033d4af207-kube-api-access-jjxnc\") pod \"kuadrant-operator-controller-manager-55467d9f7-hjbfw\" (UID: \"50c5d096-071b-432d-888b-8f033d4af207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:15:58.453974 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.453886 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw"] Jul 08 09:15:58.454147 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.454135 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:15:58.460230 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.460205 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw"] Jul 08 09:15:58.519353 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.519321 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c"] Jul 08 09:15:58.522934 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.522914 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:15:58.539275 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.539248 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c"] Jul 08 09:15:58.648249 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.648212 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afe5eed7-101d-4862-9f62-f48662a5d3f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqg5c\" (UID: \"afe5eed7-101d-4862-9f62-f48662a5d3f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:15:58.648420 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.648273 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rbcx\" (UniqueName: \"kubernetes.io/projected/afe5eed7-101d-4862-9f62-f48662a5d3f6-kube-api-access-5rbcx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqg5c\" (UID: \"afe5eed7-101d-4862-9f62-f48662a5d3f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:15:58.749382 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.749345 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afe5eed7-101d-4862-9f62-f48662a5d3f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqg5c\" (UID: \"afe5eed7-101d-4862-9f62-f48662a5d3f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:15:58.749541 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.749410 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rbcx\" (UniqueName: \"kubernetes.io/projected/afe5eed7-101d-4862-9f62-f48662a5d3f6-kube-api-access-5rbcx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqg5c\" (UID: \"afe5eed7-101d-4862-9f62-f48662a5d3f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:15:58.749739 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.749717 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afe5eed7-101d-4862-9f62-f48662a5d3f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqg5c\" (UID: \"afe5eed7-101d-4862-9f62-f48662a5d3f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:15:58.763403 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.763369 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rbcx\" (UniqueName: \"kubernetes.io/projected/afe5eed7-101d-4862-9f62-f48662a5d3f6-kube-api-access-5rbcx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqg5c\" (UID: \"afe5eed7-101d-4862-9f62-f48662a5d3f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:15:58.834390 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.834355 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:15:58.995299 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:58.995272 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c"] Jul 08 09:15:58.997813 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:15:58.997786 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe5eed7_101d_4862_9f62_f48662a5d3f6.slice/crio-98e4ecf55a0ae13b840d108c95f455ba63251320da058f82f1756c3ab518b54e WatchSource:0}: Error finding container 98e4ecf55a0ae13b840d108c95f455ba63251320da058f82f1756c3ab518b54e: Status 404 returned error can't find the container with id 98e4ecf55a0ae13b840d108c95f455ba63251320da058f82f1756c3ab518b54e Jul 08 09:15:59.395575 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:15:59.395490 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" event={"ID":"afe5eed7-101d-4862-9f62-f48662a5d3f6","Type":"ContainerStarted","Data":"98e4ecf55a0ae13b840d108c95f455ba63251320da058f82f1756c3ab518b54e"} Jul 08 09:16:01.005061 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:16:01.005004 2572 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-hjbfw_kuadrant-system_50c5d096-071b-432d-888b-8f033d4af207_0(f1a8f72030b97cd19a9dce158b5dd2d3f70eb4be8840503321be6571666a15d9): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-hjbfw to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 08 09:16:01.005407 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:16:01.005094 2572 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-hjbfw_kuadrant-system_50c5d096-071b-432d-888b-8f033d4af207_0(f1a8f72030b97cd19a9dce158b5dd2d3f70eb4be8840503321be6571666a15d9): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-hjbfw to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:16:01.404082 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:01.403999 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:16:01.411319 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:01.411290 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:16:01.414126 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:16:01.414091 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hjbfw\" is forbidden: User \"system:node:ip-10-0-136-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-68.ec2.internal' and this object" podUID="50c5d096-071b-432d-888b-8f033d4af207" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:16:01.476580 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:01.476545 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjxnc\" (UniqueName: \"kubernetes.io/projected/50c5d096-071b-432d-888b-8f033d4af207-kube-api-access-jjxnc\") pod \"50c5d096-071b-432d-888b-8f033d4af207\" (UID: \"50c5d096-071b-432d-888b-8f033d4af207\") " Jul 08 09:16:01.476813 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:01.476641 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50c5d096-071b-432d-888b-8f033d4af207-extensions-socket-volume\") pod \"50c5d096-071b-432d-888b-8f033d4af207\" (UID: \"50c5d096-071b-432d-888b-8f033d4af207\") " Jul 08 09:16:01.476965 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:01.476935 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50c5d096-071b-432d-888b-8f033d4af207-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "50c5d096-071b-432d-888b-8f033d4af207" (UID: "50c5d096-071b-432d-888b-8f033d4af207"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:16:01.478890 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:01.478865 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50c5d096-071b-432d-888b-8f033d4af207-kube-api-access-jjxnc" (OuterVolumeSpecName: "kube-api-access-jjxnc") pod "50c5d096-071b-432d-888b-8f033d4af207" (UID: "50c5d096-071b-432d-888b-8f033d4af207"). InnerVolumeSpecName "kube-api-access-jjxnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:16:01.577879 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:01.577840 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50c5d096-071b-432d-888b-8f033d4af207-extensions-socket-volume\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:16:01.577879 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:01.577872 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjxnc\" (UniqueName: \"kubernetes.io/projected/50c5d096-071b-432d-888b-8f033d4af207-kube-api-access-jjxnc\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:16:02.307594 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:02.307557 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50c5d096-071b-432d-888b-8f033d4af207" path="/var/lib/kubelet/pods/50c5d096-071b-432d-888b-8f033d4af207/volumes" Jul 08 09:16:02.308442 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:16:02.308406 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hjbfw\" is forbidden: User \"system:node:ip-10-0-136-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-68.ec2.internal' and this object" podUID="50c5d096-071b-432d-888b-8f033d4af207" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:16:02.409281 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:02.409239 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:16:02.413944 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:16:02.413910 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hjbfw\" is forbidden: User \"system:node:ip-10-0-136-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-68.ec2.internal' and this object" podUID="50c5d096-071b-432d-888b-8f033d4af207" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hjbfw" Jul 08 09:16:04.419562 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:04.419467 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" event={"ID":"afe5eed7-101d-4862-9f62-f48662a5d3f6","Type":"ContainerStarted","Data":"8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e"} Jul 08 09:16:04.419986 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:04.419577 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:16:04.440831 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:04.440778 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" podStartSLOduration=1.32478584 podStartE2EDuration="6.440761389s" totalImagesPullingTime="5.115975549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:16:04.439837112 +0000 UTC m=+562.689796394" watchObservedRunningTime="2026-07-08 09:16:04.440761389 +0000 UTC m=+562.690720663" Jul 08 09:16:15.425556 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:15.425521 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:16:22.889203 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:22.889167 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c"] Jul 08 09:16:22.889700 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:22.889448 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" podUID="afe5eed7-101d-4862-9f62-f48662a5d3f6" containerName="manager" containerID="cri-o://8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e" gracePeriod=10 Jul 08 09:16:23.436901 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.436878 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:16:23.459690 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.459665 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afe5eed7-101d-4862-9f62-f48662a5d3f6-extensions-socket-volume\") pod \"afe5eed7-101d-4862-9f62-f48662a5d3f6\" (UID: \"afe5eed7-101d-4862-9f62-f48662a5d3f6\") " Jul 08 09:16:23.459847 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.459720 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rbcx\" (UniqueName: \"kubernetes.io/projected/afe5eed7-101d-4862-9f62-f48662a5d3f6-kube-api-access-5rbcx\") pod \"afe5eed7-101d-4862-9f62-f48662a5d3f6\" (UID: \"afe5eed7-101d-4862-9f62-f48662a5d3f6\") " Jul 08 09:16:23.460160 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.460136 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afe5eed7-101d-4862-9f62-f48662a5d3f6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "afe5eed7-101d-4862-9f62-f48662a5d3f6" (UID: "afe5eed7-101d-4862-9f62-f48662a5d3f6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:16:23.461837 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.461815 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afe5eed7-101d-4862-9f62-f48662a5d3f6-kube-api-access-5rbcx" (OuterVolumeSpecName: "kube-api-access-5rbcx") pod "afe5eed7-101d-4862-9f62-f48662a5d3f6" (UID: "afe5eed7-101d-4862-9f62-f48662a5d3f6"). InnerVolumeSpecName "kube-api-access-5rbcx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:16:23.503321 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.503287 2572 generic.go:356] "Generic (PLEG): container finished" podID="afe5eed7-101d-4862-9f62-f48662a5d3f6" containerID="8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e" exitCode=0 Jul 08 09:16:23.503483 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.503335 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" Jul 08 09:16:23.503483 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.503347 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" event={"ID":"afe5eed7-101d-4862-9f62-f48662a5d3f6","Type":"ContainerDied","Data":"8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e"} Jul 08 09:16:23.503483 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.503390 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c" event={"ID":"afe5eed7-101d-4862-9f62-f48662a5d3f6","Type":"ContainerDied","Data":"98e4ecf55a0ae13b840d108c95f455ba63251320da058f82f1756c3ab518b54e"} Jul 08 09:16:23.503483 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.503412 2572 scope.go:117] "RemoveContainer" containerID="8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e" Jul 08 09:16:23.512111 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.512094 2572 scope.go:117] "RemoveContainer" containerID="8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e" Jul 08 09:16:23.512365 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:16:23.512348 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e\": container with ID starting with 8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e not found: ID does not exist" containerID="8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e" Jul 08 09:16:23.512427 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.512372 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e"} err="failed to get container status \"8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e\": rpc error: code = NotFound desc = could not find container \"8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e\": container with ID starting with 8c0e104b533c15362ccfd5512e2ce3bee1d8f27b912bf44d418a056c0e9d750e not found: ID does not exist" Jul 08 09:16:23.537489 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.537460 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c"] Jul 08 09:16:23.544717 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.544692 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqg5c"] Jul 08 09:16:23.560138 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.560107 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afe5eed7-101d-4862-9f62-f48662a5d3f6-extensions-socket-volume\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:16:23.560138 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:23.560136 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rbcx\" (UniqueName: \"kubernetes.io/projected/afe5eed7-101d-4862-9f62-f48662a5d3f6-kube-api-access-5rbcx\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:16:24.307357 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:24.307327 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afe5eed7-101d-4862-9f62-f48662a5d3f6" path="/var/lib/kubelet/pods/afe5eed7-101d-4862-9f62-f48662a5d3f6/volumes" Jul 08 09:16:42.020615 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.020574 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hx56m"] Jul 08 09:16:42.021222 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.020995 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe5eed7-101d-4862-9f62-f48662a5d3f6" containerName="manager" Jul 08 09:16:42.021222 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.021013 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe5eed7-101d-4862-9f62-f48662a5d3f6" containerName="manager" Jul 08 09:16:42.021222 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.021091 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="afe5eed7-101d-4862-9f62-f48662a5d3f6" containerName="manager" Jul 08 09:16:42.023267 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.023245 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:16:42.025858 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.025827 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 09:16:42.025858 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.025842 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 09:16:42.025858 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.025845 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rxqb9\"" Jul 08 09:16:42.026818 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.026805 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 09:16:42.036285 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.036260 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hx56m"] Jul 08 09:16:42.081202 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.081170 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrsjv\" (UniqueName: \"kubernetes.io/projected/5e1c39c3-8429-425a-91c9-102085d6d3f2-kube-api-access-vrsjv\") pod \"limitador-limitador-64c94fd64f-hx56m\" (UID: \"5e1c39c3-8429-425a-91c9-102085d6d3f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:16:42.081388 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.081228 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5e1c39c3-8429-425a-91c9-102085d6d3f2-config-file\") pod \"limitador-limitador-64c94fd64f-hx56m\" (UID: \"5e1c39c3-8429-425a-91c9-102085d6d3f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:16:42.120794 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.120739 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hx56m"] Jul 08 09:16:42.181974 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.181914 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5e1c39c3-8429-425a-91c9-102085d6d3f2-config-file\") pod \"limitador-limitador-64c94fd64f-hx56m\" (UID: \"5e1c39c3-8429-425a-91c9-102085d6d3f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:16:42.182196 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.182004 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrsjv\" (UniqueName: \"kubernetes.io/projected/5e1c39c3-8429-425a-91c9-102085d6d3f2-kube-api-access-vrsjv\") pod \"limitador-limitador-64c94fd64f-hx56m\" (UID: \"5e1c39c3-8429-425a-91c9-102085d6d3f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:16:42.184484 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.184463 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 09:16:42.189977 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.189954 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 09:16:42.193490 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.193472 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5e1c39c3-8429-425a-91c9-102085d6d3f2-config-file\") pod \"limitador-limitador-64c94fd64f-hx56m\" (UID: \"5e1c39c3-8429-425a-91c9-102085d6d3f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:16:42.200616 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.200599 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 09:16:42.211861 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.211833 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrsjv\" (UniqueName: \"kubernetes.io/projected/5e1c39c3-8429-425a-91c9-102085d6d3f2-kube-api-access-vrsjv\") pod \"limitador-limitador-64c94fd64f-hx56m\" (UID: \"5e1c39c3-8429-425a-91c9-102085d6d3f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:16:42.336711 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.336630 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rxqb9\"" Jul 08 09:16:42.345000 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.344980 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:16:42.476045 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.475996 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hx56m"] Jul 08 09:16:42.478386 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:16:42.478349 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e1c39c3_8429_425a_91c9_102085d6d3f2.slice/crio-389aafc84b336cdf92a08341610d07a0c9e1a923d56cf00be63b09d52ba7835c WatchSource:0}: Error finding container 389aafc84b336cdf92a08341610d07a0c9e1a923d56cf00be63b09d52ba7835c: Status 404 returned error can't find the container with id 389aafc84b336cdf92a08341610d07a0c9e1a923d56cf00be63b09d52ba7835c Jul 08 09:16:42.590608 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:42.590517 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" event={"ID":"5e1c39c3-8429-425a-91c9-102085d6d3f2","Type":"ContainerStarted","Data":"389aafc84b336cdf92a08341610d07a0c9e1a923d56cf00be63b09d52ba7835c"} Jul 08 09:16:45.606787 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:45.606729 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" event={"ID":"5e1c39c3-8429-425a-91c9-102085d6d3f2","Type":"ContainerStarted","Data":"00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3"} Jul 08 09:16:45.607209 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:45.606868 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:16:45.624371 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:45.624310 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" podStartSLOduration=0.985194528 podStartE2EDuration="3.624293588s" totalImagesPullingTime="2.63909906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:16:45.623606516 +0000 UTC m=+603.873565800" watchObservedRunningTime="2026-07-08 09:16:45.624293588 +0000 UTC m=+603.874252871" Jul 08 09:16:56.611610 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:56.611579 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:16:59.719057 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:59.719020 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hx56m"] Jul 08 09:16:59.719504 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:16:59.719192 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" podUID="5e1c39c3-8429-425a-91c9-102085d6d3f2" containerName="limitador" containerID="cri-o://00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3" gracePeriod=30 Jul 08 09:17:00.265990 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.265968 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:17:00.324849 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.324768 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrsjv\" (UniqueName: \"kubernetes.io/projected/5e1c39c3-8429-425a-91c9-102085d6d3f2-kube-api-access-vrsjv\") pod \"5e1c39c3-8429-425a-91c9-102085d6d3f2\" (UID: \"5e1c39c3-8429-425a-91c9-102085d6d3f2\") " Jul 08 09:17:00.324849 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.324817 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5e1c39c3-8429-425a-91c9-102085d6d3f2-config-file\") pod \"5e1c39c3-8429-425a-91c9-102085d6d3f2\" (UID: \"5e1c39c3-8429-425a-91c9-102085d6d3f2\") " Jul 08 09:17:00.330564 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.325414 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e1c39c3-8429-425a-91c9-102085d6d3f2-config-file" (OuterVolumeSpecName: "config-file") pod "5e1c39c3-8429-425a-91c9-102085d6d3f2" (UID: "5e1c39c3-8429-425a-91c9-102085d6d3f2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:17:00.330729 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.330551 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e1c39c3-8429-425a-91c9-102085d6d3f2-kube-api-access-vrsjv" (OuterVolumeSpecName: "kube-api-access-vrsjv") pod "5e1c39c3-8429-425a-91c9-102085d6d3f2" (UID: "5e1c39c3-8429-425a-91c9-102085d6d3f2"). InnerVolumeSpecName "kube-api-access-vrsjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:17:00.426251 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.426210 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrsjv\" (UniqueName: \"kubernetes.io/projected/5e1c39c3-8429-425a-91c9-102085d6d3f2-kube-api-access-vrsjv\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:17:00.426251 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.426247 2572 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5e1c39c3-8429-425a-91c9-102085d6d3f2-config-file\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:17:00.672626 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.672527 2572 generic.go:356] "Generic (PLEG): container finished" podID="5e1c39c3-8429-425a-91c9-102085d6d3f2" containerID="00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3" exitCode=0 Jul 08 09:17:00.672626 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.672575 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" event={"ID":"5e1c39c3-8429-425a-91c9-102085d6d3f2","Type":"ContainerDied","Data":"00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3"} Jul 08 09:17:00.672626 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.672596 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" Jul 08 09:17:00.672938 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.672651 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hx56m" event={"ID":"5e1c39c3-8429-425a-91c9-102085d6d3f2","Type":"ContainerDied","Data":"389aafc84b336cdf92a08341610d07a0c9e1a923d56cf00be63b09d52ba7835c"} Jul 08 09:17:00.672938 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.672674 2572 scope.go:117] "RemoveContainer" containerID="00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3" Jul 08 09:17:00.681236 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.681206 2572 scope.go:117] "RemoveContainer" containerID="00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3" Jul 08 09:17:00.681483 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:17:00.681465 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3\": container with ID starting with 00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3 not found: ID does not exist" containerID="00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3" Jul 08 09:17:00.681539 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.681491 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3"} err="failed to get container status \"00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3\": rpc error: code = NotFound desc = could not find container \"00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3\": container with ID starting with 00782d899ada511ef4154d8fde165f4be55d8139c32164b5fb04c063e18ac0f3 not found: ID does not exist" Jul 08 09:17:00.696383 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.696357 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hx56m"] Jul 08 09:17:00.700269 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:00.700248 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hx56m"] Jul 08 09:17:02.307521 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:02.307488 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e1c39c3-8429-425a-91c9-102085d6d3f2" path="/var/lib/kubelet/pods/5e1c39c3-8429-425a-91c9-102085d6d3f2/volumes" Jul 08 09:17:03.633482 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.633450 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-wjmwm"] Jul 08 09:17:03.633961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.633850 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e1c39c3-8429-425a-91c9-102085d6d3f2" containerName="limitador" Jul 08 09:17:03.633961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.633867 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e1c39c3-8429-425a-91c9-102085d6d3f2" containerName="limitador" Jul 08 09:17:03.633961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.633922 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e1c39c3-8429-425a-91c9-102085d6d3f2" containerName="limitador" Jul 08 09:17:03.636535 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.636514 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-wjmwm" Jul 08 09:17:03.639177 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.639154 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 08 09:17:03.639306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.639250 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tfgz9\"" Jul 08 09:17:03.646554 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.646520 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-wjmwm"] Jul 08 09:17:03.752573 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.752541 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6qsl\" (UniqueName: \"kubernetes.io/projected/f472b240-b1e0-4476-ab6a-646262f03444-kube-api-access-h6qsl\") pod \"postgres-69fb4594cf-wjmwm\" (UID: \"f472b240-b1e0-4476-ab6a-646262f03444\") " pod="opendatahub/postgres-69fb4594cf-wjmwm" Jul 08 09:17:03.752573 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.752577 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f472b240-b1e0-4476-ab6a-646262f03444-data\") pod \"postgres-69fb4594cf-wjmwm\" (UID: \"f472b240-b1e0-4476-ab6a-646262f03444\") " pod="opendatahub/postgres-69fb4594cf-wjmwm" Jul 08 09:17:03.853818 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.853779 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6qsl\" (UniqueName: \"kubernetes.io/projected/f472b240-b1e0-4476-ab6a-646262f03444-kube-api-access-h6qsl\") pod \"postgres-69fb4594cf-wjmwm\" (UID: \"f472b240-b1e0-4476-ab6a-646262f03444\") " pod="opendatahub/postgres-69fb4594cf-wjmwm" Jul 08 09:17:03.853818 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.853823 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f472b240-b1e0-4476-ab6a-646262f03444-data\") pod \"postgres-69fb4594cf-wjmwm\" (UID: \"f472b240-b1e0-4476-ab6a-646262f03444\") " pod="opendatahub/postgres-69fb4594cf-wjmwm" Jul 08 09:17:03.854236 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.854216 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f472b240-b1e0-4476-ab6a-646262f03444-data\") pod \"postgres-69fb4594cf-wjmwm\" (UID: \"f472b240-b1e0-4476-ab6a-646262f03444\") " pod="opendatahub/postgres-69fb4594cf-wjmwm" Jul 08 09:17:03.862689 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.862652 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6qsl\" (UniqueName: \"kubernetes.io/projected/f472b240-b1e0-4476-ab6a-646262f03444-kube-api-access-h6qsl\") pod \"postgres-69fb4594cf-wjmwm\" (UID: \"f472b240-b1e0-4476-ab6a-646262f03444\") " pod="opendatahub/postgres-69fb4594cf-wjmwm" Jul 08 09:17:03.948237 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:03.948142 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-wjmwm" Jul 08 09:17:04.073899 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:04.073804 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-wjmwm"] Jul 08 09:17:04.076539 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:17:04.076509 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf472b240_b1e0_4476_ab6a_646262f03444.slice/crio-1928246f238eba3285146a24b46a1d573de0dd39f3125f4cebc2757dac6b6e81 WatchSource:0}: Error finding container 1928246f238eba3285146a24b46a1d573de0dd39f3125f4cebc2757dac6b6e81: Status 404 returned error can't find the container with id 1928246f238eba3285146a24b46a1d573de0dd39f3125f4cebc2757dac6b6e81 Jul 08 09:17:04.695093 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:04.695006 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-wjmwm" event={"ID":"f472b240-b1e0-4476-ab6a-646262f03444","Type":"ContainerStarted","Data":"1928246f238eba3285146a24b46a1d573de0dd39f3125f4cebc2757dac6b6e81"} Jul 08 09:17:08.717101 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:08.717062 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-wjmwm" event={"ID":"f472b240-b1e0-4476-ab6a-646262f03444","Type":"ContainerStarted","Data":"64f5df736556d0bc6829ddad6315c5330a52975422d71c3f033f6e5539fb2ce4"} Jul 08 09:17:08.717561 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:08.717153 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-wjmwm" Jul 08 09:17:08.734280 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:08.734225 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-wjmwm" podStartSLOduration=1.23029955 podStartE2EDuration="5.73420389s" totalImagesPullingTime="4.50390434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:17:08.733357774 +0000 UTC m=+626.983317057" watchObservedRunningTime="2026-07-08 09:17:08.73420389 +0000 UTC m=+626.984163173" Jul 08 09:17:14.750689 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:14.750661 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-wjmwm" Jul 08 09:17:56.572846 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.572808 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-68855b66b7-krmrd"] Jul 08 09:17:56.577503 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.577479 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:56.579877 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.579848 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 08 09:17:56.580123 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.580108 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-479jv\"" Jul 08 09:17:56.588861 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.588838 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-68855b66b7-krmrd"] Jul 08 09:17:56.674845 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.674806 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-maas-api-tls\") pod \"maas-api-68855b66b7-krmrd\" (UID: \"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7\") " pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:56.675043 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.674853 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-openshift-service-ca\") pod \"maas-api-68855b66b7-krmrd\" (UID: \"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7\") " pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:56.675043 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.674944 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jl6x\" (UniqueName: \"kubernetes.io/projected/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-kube-api-access-5jl6x\") pod \"maas-api-68855b66b7-krmrd\" (UID: \"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7\") " pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:56.775631 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.775588 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-maas-api-tls\") pod \"maas-api-68855b66b7-krmrd\" (UID: \"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7\") " pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:56.775631 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.775632 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-openshift-service-ca\") pod \"maas-api-68855b66b7-krmrd\" (UID: \"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7\") " pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:56.775912 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.775727 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jl6x\" (UniqueName: \"kubernetes.io/projected/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-kube-api-access-5jl6x\") pod \"maas-api-68855b66b7-krmrd\" (UID: \"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7\") " pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:56.775912 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:17:56.775743 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 08 09:17:56.775912 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:17:56.775838 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-maas-api-tls podName:e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7 nodeName:}" failed. No retries permitted until 2026-07-08 09:17:57.275820831 +0000 UTC m=+675.525780094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-maas-api-tls") pod "maas-api-68855b66b7-krmrd" (UID: "e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7") : secret "maas-api-serving-cert" not found Jul 08 09:17:56.776402 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.776378 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-openshift-service-ca\") pod \"maas-api-68855b66b7-krmrd\" (UID: \"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7\") " pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:56.788073 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:56.788045 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jl6x\" (UniqueName: \"kubernetes.io/projected/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-kube-api-access-5jl6x\") pod \"maas-api-68855b66b7-krmrd\" (UID: \"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7\") " pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:57.211430 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.211390 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8r4k2"] Jul 08 09:17:57.214674 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.214658 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" Jul 08 09:17:57.217532 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.217512 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7985j\"" Jul 08 09:17:57.217614 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.217514 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 08 09:17:57.228887 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.228861 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8r4k2"] Jul 08 09:17:57.278506 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.278470 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-maas-api-tls\") pod \"maas-api-68855b66b7-krmrd\" (UID: \"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7\") " pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:57.278659 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.278512 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bed593dc-8139-4dbc-905d-e054ddbc591a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8r4k2\" (UID: \"bed593dc-8139-4dbc-905d-e054ddbc591a\") " pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" Jul 08 09:17:57.278728 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.278646 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxvvf\" (UniqueName: \"kubernetes.io/projected/bed593dc-8139-4dbc-905d-e054ddbc591a-kube-api-access-qxvvf\") pod \"payload-pre-processing-7964457696-8r4k2\" (UID: \"bed593dc-8139-4dbc-905d-e054ddbc591a\") " pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" Jul 08 09:17:57.280862 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.280836 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7-maas-api-tls\") pod \"maas-api-68855b66b7-krmrd\" (UID: \"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7\") " pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:57.379639 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.379601 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bed593dc-8139-4dbc-905d-e054ddbc591a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8r4k2\" (UID: \"bed593dc-8139-4dbc-905d-e054ddbc591a\") " pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" Jul 08 09:17:57.379865 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.379678 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxvvf\" (UniqueName: \"kubernetes.io/projected/bed593dc-8139-4dbc-905d-e054ddbc591a-kube-api-access-qxvvf\") pod \"payload-pre-processing-7964457696-8r4k2\" (UID: \"bed593dc-8139-4dbc-905d-e054ddbc591a\") " pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" Jul 08 09:17:57.380396 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.380370 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bed593dc-8139-4dbc-905d-e054ddbc591a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8r4k2\" (UID: \"bed593dc-8139-4dbc-905d-e054ddbc591a\") " pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" Jul 08 09:17:57.387798 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.387769 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxvvf\" (UniqueName: \"kubernetes.io/projected/bed593dc-8139-4dbc-905d-e054ddbc591a-kube-api-access-qxvvf\") pod \"payload-pre-processing-7964457696-8r4k2\" (UID: \"bed593dc-8139-4dbc-905d-e054ddbc591a\") " pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" Jul 08 09:17:57.487812 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.487771 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:57.524910 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.524877 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" Jul 08 09:17:57.647636 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.647614 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-68855b66b7-krmrd"] Jul 08 09:17:57.649687 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:17:57.649664 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6bc91fa_fb62_4171_8e85_6a7fdbdc53a7.slice/crio-349b1efc7645ec1766428daa36f760085431a4048b260c8b7de75492dc6a5bd7 WatchSource:0}: Error finding container 349b1efc7645ec1766428daa36f760085431a4048b260c8b7de75492dc6a5bd7: Status 404 returned error can't find the container with id 349b1efc7645ec1766428daa36f760085431a4048b260c8b7de75492dc6a5bd7 Jul 08 09:17:57.650961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.650946 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 09:17:57.691339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.691314 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8r4k2"] Jul 08 09:17:57.693210 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:17:57.693179 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbed593dc_8139_4dbc_905d_e054ddbc591a.slice/crio-61f6481df8735e5c8e3ad13d40f7e0128051132218f31d75362db5fb7dceefd0 WatchSource:0}: Error finding container 61f6481df8735e5c8e3ad13d40f7e0128051132218f31d75362db5fb7dceefd0: Status 404 returned error can't find the container with id 61f6481df8735e5c8e3ad13d40f7e0128051132218f31d75362db5fb7dceefd0 Jul 08 09:17:57.935663 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.935578 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-68855b66b7-krmrd" event={"ID":"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7","Type":"ContainerStarted","Data":"349b1efc7645ec1766428daa36f760085431a4048b260c8b7de75492dc6a5bd7"} Jul 08 09:17:57.936772 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:57.936726 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" event={"ID":"bed593dc-8139-4dbc-905d-e054ddbc591a","Type":"ContainerStarted","Data":"61f6481df8735e5c8e3ad13d40f7e0128051132218f31d75362db5fb7dceefd0"} Jul 08 09:17:59.948632 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:59.948539 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" event={"ID":"bed593dc-8139-4dbc-905d-e054ddbc591a","Type":"ContainerStarted","Data":"0d0e140f0d3e0e115574fdebf07037944870a4aff3f1033274f32c17887fe436"} Jul 08 09:17:59.948632 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:59.948607 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" Jul 08 09:17:59.950050 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:59.950028 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-68855b66b7-krmrd" event={"ID":"e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7","Type":"ContainerStarted","Data":"555225accac775aa580a8c6b1c81c91b69406a338a0ce627d8b0bb633d38677b"} Jul 08 09:17:59.950164 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:59.950139 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:17:59.972570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:17:59.972525 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" podStartSLOduration=1.5193416549999998 podStartE2EDuration="2.972509557s" totalImagesPullingTime="1.453167902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:17:59.970253334 +0000 UTC m=+678.220212614" watchObservedRunningTime="2026-07-08 09:17:59.972509557 +0000 UTC m=+678.222468871" Jul 08 09:18:05.960644 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:05.960615 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-68855b66b7-krmrd" Jul 08 09:18:05.977171 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:05.977104 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-68855b66b7-krmrd" podStartSLOduration=8.024322014 podStartE2EDuration="9.977085461s" totalImagesPullingTime="1.952763447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:17:59.988523108 +0000 UTC m=+678.238482392" watchObservedRunningTime="2026-07-08 09:18:05.977085461 +0000 UTC m=+684.227044745" Jul 08 09:18:10.957559 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:10.957521 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-8r4k2" Jul 08 09:18:20.280557 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.280521 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk"] Jul 08 09:18:20.283828 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.283813 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.287358 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.287331 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 09:18:20.287482 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.287369 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 09:18:20.287482 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.287387 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 09:18:20.287482 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.287460 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w2rf8\"" Jul 08 09:18:20.293366 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.293346 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk"] Jul 08 09:18:20.366721 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.366685 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.366721 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.366728 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b77d4eeb-046b-4857-b4dc-3d57112eab56-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.366934 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.366784 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.366934 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.366802 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.366934 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.366895 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4b7q\" (UniqueName: \"kubernetes.io/projected/b77d4eeb-046b-4857-b4dc-3d57112eab56-kube-api-access-k4b7q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.367033 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.366930 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.467679 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.467644 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.467679 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.467677 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.467969 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.467698 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4b7q\" (UniqueName: \"kubernetes.io/projected/b77d4eeb-046b-4857-b4dc-3d57112eab56-kube-api-access-k4b7q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.467969 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.467722 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.467969 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.467920 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.468116 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.467977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b77d4eeb-046b-4857-b4dc-3d57112eab56-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.468116 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.468083 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.468116 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.468110 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.468217 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.468186 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.469983 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.469960 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b77d4eeb-046b-4857-b4dc-3d57112eab56-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.470337 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.470315 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b77d4eeb-046b-4857-b4dc-3d57112eab56-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.475799 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.475774 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4b7q\" (UniqueName: \"kubernetes.io/projected/b77d4eeb-046b-4857-b4dc-3d57112eab56-kube-api-access-k4b7q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qmxfk\" (UID: \"b77d4eeb-046b-4857-b4dc-3d57112eab56\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.594946 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.594846 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:20.728278 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:20.728254 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk"] Jul 08 09:18:20.730696 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:18:20.730666 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb77d4eeb_046b_4857_b4dc_3d57112eab56.slice/crio-ebe80dcfb818a0b5616967eaaee528d01805c4fb1c1b80a17053214dcd52137b WatchSource:0}: Error finding container ebe80dcfb818a0b5616967eaaee528d01805c4fb1c1b80a17053214dcd52137b: Status 404 returned error can't find the container with id ebe80dcfb818a0b5616967eaaee528d01805c4fb1c1b80a17053214dcd52137b Jul 08 09:18:21.050570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:21.050529 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" event={"ID":"b77d4eeb-046b-4857-b4dc-3d57112eab56","Type":"ContainerStarted","Data":"ebe80dcfb818a0b5616967eaaee528d01805c4fb1c1b80a17053214dcd52137b"} Jul 08 09:18:22.057896 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:22.057802 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" event={"ID":"b77d4eeb-046b-4857-b4dc-3d57112eab56","Type":"ContainerStarted","Data":"2fd790fd31c985aae4715506e8a686379571b7c785a35bfebc6eb47c739ff824"} Jul 08 09:18:22.075060 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:22.075007 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" podStartSLOduration=1.071900299 podStartE2EDuration="2.07499158s" totalImagesPullingTime="1.003091281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:18:22.074319956 +0000 UTC m=+700.324279232" watchObservedRunningTime="2026-07-08 09:18:22.07499158 +0000 UTC m=+700.324950864" Jul 08 09:18:30.596056 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:30.596012 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:30.596561 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:30.596091 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:30.608517 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:30.608491 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:31.114391 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:31.114367 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qmxfk" Jul 08 09:18:42.310337 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.310271 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782"] Jul 08 09:18:42.313967 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.313945 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.317829 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.317808 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 08 09:18:42.334086 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.334050 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782"] Jul 08 09:18:42.453297 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.453264 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.453297 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.453303 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.453560 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.453426 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.453560 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.453476 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.453560 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.453525 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc9vj\" (UniqueName: \"kubernetes.io/projected/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-kube-api-access-hc9vj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.453701 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.453586 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.554779 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.554721 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.554974 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.554792 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9vj\" (UniqueName: \"kubernetes.io/projected/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-kube-api-access-hc9vj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.554974 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.554840 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.554974 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.554912 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.554974 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.554942 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.555181 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.554977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.555181 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.555126 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.555297 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.555271 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.555356 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.555344 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.557114 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.557097 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.557303 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.557276 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.565318 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.565256 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc9vj\" (UniqueName: \"kubernetes.io/projected/9b21d8ba-7db2-40d9-a964-204f0a3ac8b4-kube-api-access-hc9vj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782\" (UID: \"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.624011 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.623975 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:42.762072 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:42.762049 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782"] Jul 08 09:18:42.763803 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:18:42.763778 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b21d8ba_7db2_40d9_a964_204f0a3ac8b4.slice/crio-82dbd304b05eccf3cc32e1fae1aec4a8ad4b8fb33ad256b5cfbac11ad875b5a5 WatchSource:0}: Error finding container 82dbd304b05eccf3cc32e1fae1aec4a8ad4b8fb33ad256b5cfbac11ad875b5a5: Status 404 returned error can't find the container with id 82dbd304b05eccf3cc32e1fae1aec4a8ad4b8fb33ad256b5cfbac11ad875b5a5 Jul 08 09:18:43.157176 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:43.157122 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" event={"ID":"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4","Type":"ContainerStarted","Data":"bcff6df911a93c7d3d41caa3b70ac939b6f73a09f8985cfd9dcd8402ca417151"} Jul 08 09:18:43.157369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:43.157187 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" event={"ID":"9b21d8ba-7db2-40d9-a964-204f0a3ac8b4","Type":"ContainerStarted","Data":"82dbd304b05eccf3cc32e1fae1aec4a8ad4b8fb33ad256b5cfbac11ad875b5a5"} Jul 08 09:18:43.192087 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:43.192034 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" podStartSLOduration=1.013073658 podStartE2EDuration="1.192019966s" totalImagesPullingTime="178.946308ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:18:43.191395514 +0000 UTC m=+721.441354797" watchObservedRunningTime="2026-07-08 09:18:43.192019966 +0000 UTC m=+721.441979250" Jul 08 09:18:52.624505 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:52.624469 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:52.624909 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:52.624536 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:52.637068 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:52.637040 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:18:53.215708 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:18:53.215679 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782" Jul 08 09:32:58.270112 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.270072 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5"] Jul 08 09:32:58.273396 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.273374 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.276451 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.276432 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-8294e149-serving-cert\"" Jul 08 09:32:58.297051 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.297022 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5"] Jul 08 09:32:58.429244 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.429207 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b1899c-5871-476f-b8ce-e7cc44201400-openshift-service-ca\") pod \"maas-api-e2e-ait-8294e149-545684dbcb-xdtb5\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.429436 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.429282 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5b1899c-5871-476f-b8ce-e7cc44201400-maas-api-tls\") pod \"maas-api-e2e-ait-8294e149-545684dbcb-xdtb5\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.429436 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.429307 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5rwt\" (UniqueName: \"kubernetes.io/projected/c5b1899c-5871-476f-b8ce-e7cc44201400-kube-api-access-p5rwt\") pod \"maas-api-e2e-ait-8294e149-545684dbcb-xdtb5\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.530724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.530637 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b1899c-5871-476f-b8ce-e7cc44201400-openshift-service-ca\") pod \"maas-api-e2e-ait-8294e149-545684dbcb-xdtb5\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.530724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.530687 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5b1899c-5871-476f-b8ce-e7cc44201400-maas-api-tls\") pod \"maas-api-e2e-ait-8294e149-545684dbcb-xdtb5\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.530724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.530713 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5rwt\" (UniqueName: \"kubernetes.io/projected/c5b1899c-5871-476f-b8ce-e7cc44201400-kube-api-access-p5rwt\") pod \"maas-api-e2e-ait-8294e149-545684dbcb-xdtb5\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.531310 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.531283 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b1899c-5871-476f-b8ce-e7cc44201400-openshift-service-ca\") pod \"maas-api-e2e-ait-8294e149-545684dbcb-xdtb5\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.533107 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.533086 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5b1899c-5871-476f-b8ce-e7cc44201400-maas-api-tls\") pod \"maas-api-e2e-ait-8294e149-545684dbcb-xdtb5\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.545081 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.545056 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5rwt\" (UniqueName: \"kubernetes.io/projected/c5b1899c-5871-476f-b8ce-e7cc44201400-kube-api-access-p5rwt\") pod \"maas-api-e2e-ait-8294e149-545684dbcb-xdtb5\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.582992 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.582954 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:32:58.777642 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.777555 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5"] Jul 08 09:32:58.781128 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:32:58.781075 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b1899c_5871_476f_b8ce_e7cc44201400.slice/crio-bcead90775c739dcfd983481c0f53eecf8c8f693ae8d43831befcc24c7f0103d WatchSource:0}: Error finding container bcead90775c739dcfd983481c0f53eecf8c8f693ae8d43831befcc24c7f0103d: Status 404 returned error can't find the container with id bcead90775c739dcfd983481c0f53eecf8c8f693ae8d43831befcc24c7f0103d Jul 08 09:32:58.782579 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:58.782561 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 09:32:59.020491 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:32:59.020442 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" event={"ID":"c5b1899c-5871-476f-b8ce-e7cc44201400","Type":"ContainerStarted","Data":"bcead90775c739dcfd983481c0f53eecf8c8f693ae8d43831befcc24c7f0103d"} Jul 08 09:33:00.026114 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:00.026077 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" event={"ID":"c5b1899c-5871-476f-b8ce-e7cc44201400","Type":"ContainerStarted","Data":"e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138"} Jul 08 09:33:00.026512 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:00.026171 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:33:00.099417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:00.099362 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" podStartSLOduration=1.493277502 podStartE2EDuration="2.099345515s" totalImagesPullingTime="606.068013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:33:00.096921453 +0000 UTC m=+1578.346880726" watchObservedRunningTime="2026-07-08 09:33:00.099345515 +0000 UTC m=+1578.349304809" Jul 08 09:33:03.736226 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:03.736192 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5"] Jul 08 09:33:03.736621 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:03.736374 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" podUID="c5b1899c-5871-476f-b8ce-e7cc44201400" containerName="maas-api" containerID="cri-o://e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138" gracePeriod=30 Jul 08 09:33:03.989431 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:03.989374 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:33:04.046591 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.046556 2572 generic.go:356] "Generic (PLEG): container finished" podID="c5b1899c-5871-476f-b8ce-e7cc44201400" containerID="e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138" exitCode=0 Jul 08 09:33:04.046769 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.046604 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" Jul 08 09:33:04.046769 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.046652 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" event={"ID":"c5b1899c-5871-476f-b8ce-e7cc44201400","Type":"ContainerDied","Data":"e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138"} Jul 08 09:33:04.046769 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.046713 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5" event={"ID":"c5b1899c-5871-476f-b8ce-e7cc44201400","Type":"ContainerDied","Data":"bcead90775c739dcfd983481c0f53eecf8c8f693ae8d43831befcc24c7f0103d"} Jul 08 09:33:04.046769 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.046742 2572 scope.go:117] "RemoveContainer" containerID="e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138" Jul 08 09:33:04.055458 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.055443 2572 scope.go:117] "RemoveContainer" containerID="e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138" Jul 08 09:33:04.055699 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:33:04.055683 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138\": container with ID starting with e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138 not found: ID does not exist" containerID="e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138" Jul 08 09:33:04.055741 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.055708 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138"} err="failed to get container status \"e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138\": rpc error: code = NotFound desc = could not find container \"e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138\": container with ID starting with e3972d5bbaaa3670ad796da55be1f8463c7a393d0ab629f885f8d0d8b26d6138 not found: ID does not exist" Jul 08 09:33:04.186596 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.186565 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5b1899c-5871-476f-b8ce-e7cc44201400-maas-api-tls\") pod \"c5b1899c-5871-476f-b8ce-e7cc44201400\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " Jul 08 09:33:04.186596 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.186602 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5rwt\" (UniqueName: \"kubernetes.io/projected/c5b1899c-5871-476f-b8ce-e7cc44201400-kube-api-access-p5rwt\") pod \"c5b1899c-5871-476f-b8ce-e7cc44201400\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " Jul 08 09:33:04.186869 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.186709 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b1899c-5871-476f-b8ce-e7cc44201400-openshift-service-ca\") pod \"c5b1899c-5871-476f-b8ce-e7cc44201400\" (UID: \"c5b1899c-5871-476f-b8ce-e7cc44201400\") " Jul 08 09:33:04.192444 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.191438 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5b1899c-5871-476f-b8ce-e7cc44201400-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5b1899c-5871-476f-b8ce-e7cc44201400" (UID: "c5b1899c-5871-476f-b8ce-e7cc44201400"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:33:04.193065 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.193042 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5b1899c-5871-476f-b8ce-e7cc44201400-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5b1899c-5871-476f-b8ce-e7cc44201400" (UID: "c5b1899c-5871-476f-b8ce-e7cc44201400"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:33:04.194713 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.194691 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5b1899c-5871-476f-b8ce-e7cc44201400-kube-api-access-p5rwt" (OuterVolumeSpecName: "kube-api-access-p5rwt") pod "c5b1899c-5871-476f-b8ce-e7cc44201400" (UID: "c5b1899c-5871-476f-b8ce-e7cc44201400"). InnerVolumeSpecName "kube-api-access-p5rwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:33:04.288170 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.288076 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b1899c-5871-476f-b8ce-e7cc44201400-openshift-service-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:04.288170 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.288107 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5b1899c-5871-476f-b8ce-e7cc44201400-maas-api-tls\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:04.288170 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.288120 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5rwt\" (UniqueName: \"kubernetes.io/projected/c5b1899c-5871-476f-b8ce-e7cc44201400-kube-api-access-p5rwt\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:04.369943 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.369910 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5"] Jul 08 09:33:04.376408 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:04.376384 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-8294e149-545684dbcb-xdtb5"] Jul 08 09:33:06.307764 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:06.307722 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5b1899c-5871-476f-b8ce-e7cc44201400" path="/var/lib/kubelet/pods/c5b1899c-5871-476f-b8ce-e7cc44201400/volumes" Jul 08 09:33:11.323596 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.323556 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh"] Jul 08 09:33:11.324104 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.324024 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5b1899c-5871-476f-b8ce-e7cc44201400" containerName="maas-api" Jul 08 09:33:11.324104 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.324043 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5b1899c-5871-476f-b8ce-e7cc44201400" containerName="maas-api" Jul 08 09:33:11.324215 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.324114 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5b1899c-5871-476f-b8ce-e7cc44201400" containerName="maas-api" Jul 08 09:33:11.328394 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.328371 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:11.331357 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.331333 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-492b5fbf-serving-cert\"" Jul 08 09:33:11.344786 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.344728 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8889edb0-9eb7-49c6-99d9-a913c0f3a274-openshift-service-ca\") pod \"maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:11.344786 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.344774 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8889edb0-9eb7-49c6-99d9-a913c0f3a274-maas-api-tls\") pod \"maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:11.344976 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.344867 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j2sv\" (UniqueName: \"kubernetes.io/projected/8889edb0-9eb7-49c6-99d9-a913c0f3a274-kube-api-access-9j2sv\") pod \"maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:11.358321 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.358295 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh"] Jul 08 09:33:11.445895 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.445865 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8889edb0-9eb7-49c6-99d9-a913c0f3a274-openshift-service-ca\") pod \"maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:11.446078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.445901 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8889edb0-9eb7-49c6-99d9-a913c0f3a274-maas-api-tls\") pod \"maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:11.446078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.445975 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j2sv\" (UniqueName: \"kubernetes.io/projected/8889edb0-9eb7-49c6-99d9-a913c0f3a274-kube-api-access-9j2sv\") pod \"maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:11.446149 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:33:11.446089 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-492b5fbf-serving-cert: secret "maas-api-e2e-ait-492b5fbf-serving-cert" not found Jul 08 09:33:11.446185 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:33:11.446157 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8889edb0-9eb7-49c6-99d9-a913c0f3a274-maas-api-tls podName:8889edb0-9eb7-49c6-99d9-a913c0f3a274 nodeName:}" failed. No retries permitted until 2026-07-08 09:33:11.946139469 +0000 UTC m=+1590.196098747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8889edb0-9eb7-49c6-99d9-a913c0f3a274-maas-api-tls") pod "maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" (UID: "8889edb0-9eb7-49c6-99d9-a913c0f3a274") : secret "maas-api-e2e-ait-492b5fbf-serving-cert" not found Jul 08 09:33:11.446640 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.446621 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8889edb0-9eb7-49c6-99d9-a913c0f3a274-openshift-service-ca\") pod \"maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:11.464191 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.464162 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j2sv\" (UniqueName: \"kubernetes.io/projected/8889edb0-9eb7-49c6-99d9-a913c0f3a274-kube-api-access-9j2sv\") pod \"maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:11.951010 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.950967 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8889edb0-9eb7-49c6-99d9-a913c0f3a274-maas-api-tls\") pod \"maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:11.953368 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:11.953338 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8889edb0-9eb7-49c6-99d9-a913c0f3a274-maas-api-tls\") pod \"maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:12.239591 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:12.239549 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:12.394991 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:12.394957 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh"] Jul 08 09:33:12.399425 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:33:12.399391 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8889edb0_9eb7_49c6_99d9_a913c0f3a274.slice/crio-3a96bc012ce4ad764e2e4a7b7f2eda2fefd2bc02c42e84fcf295a63f3403daff WatchSource:0}: Error finding container 3a96bc012ce4ad764e2e4a7b7f2eda2fefd2bc02c42e84fcf295a63f3403daff: Status 404 returned error can't find the container with id 3a96bc012ce4ad764e2e4a7b7f2eda2fefd2bc02c42e84fcf295a63f3403daff Jul 08 09:33:13.097609 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:13.097571 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" event={"ID":"8889edb0-9eb7-49c6-99d9-a913c0f3a274","Type":"ContainerStarted","Data":"ce0425b4564f07e99331a67caf6deed6d0c3b6a15245f7c39e55397df87d9962"} Jul 08 09:33:13.097802 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:13.097621 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" event={"ID":"8889edb0-9eb7-49c6-99d9-a913c0f3a274","Type":"ContainerStarted","Data":"3a96bc012ce4ad764e2e4a7b7f2eda2fefd2bc02c42e84fcf295a63f3403daff"} Jul 08 09:33:13.097802 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:13.097708 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:13.145691 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:13.145641 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" podStartSLOduration=1.629231699 podStartE2EDuration="2.145624555s" totalImagesPullingTime="516.392856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:33:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:33:13.145525326 +0000 UTC m=+1591.395484609" watchObservedRunningTime="2026-07-08 09:33:13.145624555 +0000 UTC m=+1591.395583837" Jul 08 09:33:16.908787 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:16.908741 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh"] Jul 08 09:33:16.909200 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:16.908967 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" podUID="8889edb0-9eb7-49c6-99d9-a913c0f3a274" containerName="maas-api" containerID="cri-o://ce0425b4564f07e99331a67caf6deed6d0c3b6a15245f7c39e55397df87d9962" gracePeriod=30 Jul 08 09:33:17.117095 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.117060 2572 generic.go:356] "Generic (PLEG): container finished" podID="8889edb0-9eb7-49c6-99d9-a913c0f3a274" containerID="ce0425b4564f07e99331a67caf6deed6d0c3b6a15245f7c39e55397df87d9962" exitCode=0 Jul 08 09:33:17.117266 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.117131 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" event={"ID":"8889edb0-9eb7-49c6-99d9-a913c0f3a274","Type":"ContainerDied","Data":"ce0425b4564f07e99331a67caf6deed6d0c3b6a15245f7c39e55397df87d9962"} Jul 08 09:33:17.153425 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.153397 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:17.197780 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.197687 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8889edb0-9eb7-49c6-99d9-a913c0f3a274-openshift-service-ca\") pod \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " Jul 08 09:33:17.197780 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.197739 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8889edb0-9eb7-49c6-99d9-a913c0f3a274-maas-api-tls\") pod \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " Jul 08 09:33:17.197948 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.197809 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j2sv\" (UniqueName: \"kubernetes.io/projected/8889edb0-9eb7-49c6-99d9-a913c0f3a274-kube-api-access-9j2sv\") pod \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\" (UID: \"8889edb0-9eb7-49c6-99d9-a913c0f3a274\") " Jul 08 09:33:17.198039 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.198009 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8889edb0-9eb7-49c6-99d9-a913c0f3a274-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8889edb0-9eb7-49c6-99d9-a913c0f3a274" (UID: "8889edb0-9eb7-49c6-99d9-a913c0f3a274"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:33:17.198177 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.198089 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8889edb0-9eb7-49c6-99d9-a913c0f3a274-openshift-service-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:17.200010 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.199985 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8889edb0-9eb7-49c6-99d9-a913c0f3a274-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8889edb0-9eb7-49c6-99d9-a913c0f3a274" (UID: "8889edb0-9eb7-49c6-99d9-a913c0f3a274"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:33:17.200101 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.200026 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8889edb0-9eb7-49c6-99d9-a913c0f3a274-kube-api-access-9j2sv" (OuterVolumeSpecName: "kube-api-access-9j2sv") pod "8889edb0-9eb7-49c6-99d9-a913c0f3a274" (UID: "8889edb0-9eb7-49c6-99d9-a913c0f3a274"). InnerVolumeSpecName "kube-api-access-9j2sv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:33:17.298843 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.298805 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8889edb0-9eb7-49c6-99d9-a913c0f3a274-maas-api-tls\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:17.298843 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:17.298833 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j2sv\" (UniqueName: \"kubernetes.io/projected/8889edb0-9eb7-49c6-99d9-a913c0f3a274-kube-api-access-9j2sv\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:18.122889 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:18.122854 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" event={"ID":"8889edb0-9eb7-49c6-99d9-a913c0f3a274","Type":"ContainerDied","Data":"3a96bc012ce4ad764e2e4a7b7f2eda2fefd2bc02c42e84fcf295a63f3403daff"} Jul 08 09:33:18.123299 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:18.122909 2572 scope.go:117] "RemoveContainer" containerID="ce0425b4564f07e99331a67caf6deed6d0c3b6a15245f7c39e55397df87d9962" Jul 08 09:33:18.123299 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:18.122865 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh" Jul 08 09:33:18.154557 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:18.154518 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh"] Jul 08 09:33:18.166337 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:18.166310 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-492b5fbf-8574bf5f66-glkqh"] Jul 08 09:33:18.307847 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:18.307813 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8889edb0-9eb7-49c6-99d9-a913c0f3a274" path="/var/lib/kubelet/pods/8889edb0-9eb7-49c6-99d9-a913c0f3a274/volumes" Jul 08 09:33:29.864460 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:29.864421 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft"] Jul 08 09:33:29.864898 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:29.864883 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8889edb0-9eb7-49c6-99d9-a913c0f3a274" containerName="maas-api" Jul 08 09:33:29.864945 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:29.864900 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8889edb0-9eb7-49c6-99d9-a913c0f3a274" containerName="maas-api" Jul 08 09:33:29.865014 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:29.865002 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8889edb0-9eb7-49c6-99d9-a913c0f3a274" containerName="maas-api" Jul 08 09:33:29.870209 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:29.870183 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:29.875355 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:29.875330 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2b37457f-openshift-default-dockercfg-smkmr\"" Jul 08 09:33:29.898305 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:29.898273 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft"] Jul 08 09:33:30.014153 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.014114 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prr9n\" (UniqueName: \"kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-kube-api-access-prr9n\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.014153 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.014155 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-data\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.014369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.014178 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-credential-socket\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.014369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.014206 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-podinfo\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.014369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.014235 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-envoy\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.014369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.014255 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-token\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.014369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.014314 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-socket\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.014369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.014345 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-certs\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.014369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.014368 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13048f76-e2ff-4da9-b4fd-438c1b188a97-istiod-ca-cert\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.115499 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.115410 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-podinfo\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.115499 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.115458 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-envoy\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.115499 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.115482 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-token\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.115742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.115518 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-socket\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.115742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.115539 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-certs\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.115742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.115556 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13048f76-e2ff-4da9-b4fd-438c1b188a97-istiod-ca-cert\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.115742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.115603 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prr9n\" (UniqueName: \"kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-kube-api-access-prr9n\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.115742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.115632 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-data\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.115742 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.115659 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-credential-socket\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.116056 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.115985 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-socket\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.116056 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.116028 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-certs\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.116265 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.116245 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-data\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.116338 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.116265 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-credential-socket\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.116395 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.116360 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13048f76-e2ff-4da9-b4fd-438c1b188a97-istiod-ca-cert\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.118318 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.118289 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-envoy\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.118453 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.118436 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-podinfo\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.126637 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.126608 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-token\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.127525 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.127508 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prr9n\" (UniqueName: \"kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-kube-api-access-prr9n\") pod \"e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.180978 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.180948 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:30.328543 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:30.325687 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft"] Jul 08 09:33:30.332201 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:33:30.332164 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13048f76_e2ff_4da9_b4fd_438c1b188a97.slice/crio-81aaceb6aad4c5fd633e16ceb506fd6176b04a992010cd1864252e8275941f04 WatchSource:0}: Error finding container 81aaceb6aad4c5fd633e16ceb506fd6176b04a992010cd1864252e8275941f04: Status 404 returned error can't find the container with id 81aaceb6aad4c5fd633e16ceb506fd6176b04a992010cd1864252e8275941f04 Jul 08 09:33:31.187240 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.187193 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" event={"ID":"13048f76-e2ff-4da9-b4fd-438c1b188a97","Type":"ContainerStarted","Data":"81aaceb6aad4c5fd633e16ceb506fd6176b04a992010cd1864252e8275941f04"} Jul 08 09:33:31.200131 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.200091 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh"] Jul 08 09:33:31.204069 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.204045 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:31.209016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.208989 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-2b37457f-serving-cert\"" Jul 08 09:33:31.234731 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.234687 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh"] Jul 08 09:33:31.326746 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.326712 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chmmv\" (UniqueName: \"kubernetes.io/projected/46773e2b-90b5-4b03-a9e0-130823581f65-kube-api-access-chmmv\") pod \"maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:31.326950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.326784 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46773e2b-90b5-4b03-a9e0-130823581f65-openshift-service-ca\") pod \"maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:31.326950 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.326908 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46773e2b-90b5-4b03-a9e0-130823581f65-maas-api-tls\") pod \"maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:31.427692 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.427655 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chmmv\" (UniqueName: \"kubernetes.io/projected/46773e2b-90b5-4b03-a9e0-130823581f65-kube-api-access-chmmv\") pod \"maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:31.427885 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.427703 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46773e2b-90b5-4b03-a9e0-130823581f65-openshift-service-ca\") pod \"maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:31.427885 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.427800 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46773e2b-90b5-4b03-a9e0-130823581f65-maas-api-tls\") pod \"maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:31.427966 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:33:31.427895 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-2b37457f-serving-cert: secret "maas-api-e2e-derive-2b37457f-serving-cert" not found Jul 08 09:33:31.427966 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:33:31.427946 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46773e2b-90b5-4b03-a9e0-130823581f65-maas-api-tls podName:46773e2b-90b5-4b03-a9e0-130823581f65 nodeName:}" failed. No retries permitted until 2026-07-08 09:33:31.927930726 +0000 UTC m=+1610.177889987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/46773e2b-90b5-4b03-a9e0-130823581f65-maas-api-tls") pod "maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" (UID: "46773e2b-90b5-4b03-a9e0-130823581f65") : secret "maas-api-e2e-derive-2b37457f-serving-cert" not found Jul 08 09:33:31.428410 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.428391 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46773e2b-90b5-4b03-a9e0-130823581f65-openshift-service-ca\") pod \"maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:31.446563 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.446479 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chmmv\" (UniqueName: \"kubernetes.io/projected/46773e2b-90b5-4b03-a9e0-130823581f65-kube-api-access-chmmv\") pod \"maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:31.932057 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.931998 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46773e2b-90b5-4b03-a9e0-130823581f65-maas-api-tls\") pod \"maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:31.935046 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:31.935007 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46773e2b-90b5-4b03-a9e0-130823581f65-maas-api-tls\") pod \"maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:32.115231 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:32.115199 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:32.296976 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:32.296947 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh"] Jul 08 09:33:33.237070 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:33:33.237038 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46773e2b_90b5_4b03_a9e0_130823581f65.slice/crio-c58f8536a7eb92eeac2035a5012a63986a8c9e237246cea87ac24837a66dfd01 WatchSource:0}: Error finding container c58f8536a7eb92eeac2035a5012a63986a8c9e237246cea87ac24837a66dfd01: Status 404 returned error can't find the container with id c58f8536a7eb92eeac2035a5012a63986a8c9e237246cea87ac24837a66dfd01 Jul 08 09:33:33.299395 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:33.299356 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:33:33.299667 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:33.299434 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:33:33.299667 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:33.299463 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:33:34.206288 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:34.206252 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" event={"ID":"46773e2b-90b5-4b03-a9e0-130823581f65","Type":"ContainerStarted","Data":"370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab"} Jul 08 09:33:34.206495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:34.206311 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" event={"ID":"46773e2b-90b5-4b03-a9e0-130823581f65","Type":"ContainerStarted","Data":"c58f8536a7eb92eeac2035a5012a63986a8c9e237246cea87ac24837a66dfd01"} Jul 08 09:33:34.206495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:34.206336 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:34.207778 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:34.207741 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" event={"ID":"13048f76-e2ff-4da9-b4fd-438c1b188a97","Type":"ContainerStarted","Data":"0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6"} Jul 08 09:33:34.227110 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:34.227062 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" podStartSLOduration=2.792855607 podStartE2EDuration="3.227046149s" totalImagesPullingTime="434.190542ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:33:34.225274974 +0000 UTC m=+1612.475234251" watchObservedRunningTime="2026-07-08 09:33:34.227046149 +0000 UTC m=+1612.477005430" Jul 08 09:33:34.246614 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:34.246571 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" podStartSLOduration=2.281399724 podStartE2EDuration="5.246554633s" totalImagesPullingTime="2.965154909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:33:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:33:34.24442659 +0000 UTC m=+1612.494385873" watchObservedRunningTime="2026-07-08 09:33:34.246554633 +0000 UTC m=+1612.496513914" Jul 08 09:33:35.181424 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:35.181380 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:35.186491 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:35.186466 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:35.214260 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:35.214230 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:35.215179 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:35.215158 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:35.649617 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:35.649575 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft"] Jul 08 09:33:35.992183 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:35.992149 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh"] Jul 08 09:33:36.220174 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.220120 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" podUID="46773e2b-90b5-4b03-a9e0-130823581f65" containerName="maas-api" containerID="cri-o://370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab" gracePeriod=30 Jul 08 09:33:36.220634 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.220198 2572 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" secret="" err="secret \"e2e-derive-2b37457f-openshift-default-dockercfg-smkmr\" not found" Jul 08 09:33:36.465570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.465546 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:36.573469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.573434 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46773e2b-90b5-4b03-a9e0-130823581f65-maas-api-tls\") pod \"46773e2b-90b5-4b03-a9e0-130823581f65\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " Jul 08 09:33:36.573717 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.573480 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chmmv\" (UniqueName: \"kubernetes.io/projected/46773e2b-90b5-4b03-a9e0-130823581f65-kube-api-access-chmmv\") pod \"46773e2b-90b5-4b03-a9e0-130823581f65\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " Jul 08 09:33:36.573717 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.573523 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46773e2b-90b5-4b03-a9e0-130823581f65-openshift-service-ca\") pod \"46773e2b-90b5-4b03-a9e0-130823581f65\" (UID: \"46773e2b-90b5-4b03-a9e0-130823581f65\") " Jul 08 09:33:36.573946 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.573922 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46773e2b-90b5-4b03-a9e0-130823581f65-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46773e2b-90b5-4b03-a9e0-130823581f65" (UID: "46773e2b-90b5-4b03-a9e0-130823581f65"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:33:36.575545 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.575519 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46773e2b-90b5-4b03-a9e0-130823581f65-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46773e2b-90b5-4b03-a9e0-130823581f65" (UID: "46773e2b-90b5-4b03-a9e0-130823581f65"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:33:36.575636 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.575558 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46773e2b-90b5-4b03-a9e0-130823581f65-kube-api-access-chmmv" (OuterVolumeSpecName: "kube-api-access-chmmv") pod "46773e2b-90b5-4b03-a9e0-130823581f65" (UID: "46773e2b-90b5-4b03-a9e0-130823581f65"). InnerVolumeSpecName "kube-api-access-chmmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:33:36.674622 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.674534 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46773e2b-90b5-4b03-a9e0-130823581f65-maas-api-tls\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:36.674622 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.674565 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chmmv\" (UniqueName: \"kubernetes.io/projected/46773e2b-90b5-4b03-a9e0-130823581f65-kube-api-access-chmmv\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:36.674622 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:36.674575 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46773e2b-90b5-4b03-a9e0-130823581f65-openshift-service-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:37.226094 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:37.226053 2572 generic.go:356] "Generic (PLEG): container finished" podID="46773e2b-90b5-4b03-a9e0-130823581f65" containerID="370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab" exitCode=0 Jul 08 09:33:37.226538 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:37.226106 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" Jul 08 09:33:37.226538 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:37.226137 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" event={"ID":"46773e2b-90b5-4b03-a9e0-130823581f65","Type":"ContainerDied","Data":"370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab"} Jul 08 09:33:37.226538 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:37.226192 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh" event={"ID":"46773e2b-90b5-4b03-a9e0-130823581f65","Type":"ContainerDied","Data":"c58f8536a7eb92eeac2035a5012a63986a8c9e237246cea87ac24837a66dfd01"} Jul 08 09:33:37.226538 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:37.226214 2572 scope.go:117] "RemoveContainer" containerID="370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab" Jul 08 09:33:37.226538 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:37.226431 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" podUID="13048f76-e2ff-4da9-b4fd-438c1b188a97" containerName="istio-proxy" containerID="cri-o://0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6" gracePeriod=30 Jul 08 09:33:37.235576 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:37.235556 2572 scope.go:117] "RemoveContainer" containerID="370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab" Jul 08 09:33:37.235917 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:33:37.235895 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab\": container with ID starting with 370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab not found: ID does not exist" containerID="370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab" Jul 08 09:33:37.235972 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:37.235926 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab"} err="failed to get container status \"370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab\": rpc error: code = NotFound desc = could not find container \"370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab\": container with ID starting with 370cb1ca68a97a1b92f6a9964f768bb9b183726ec91d2df1c1f88af8fc3d61ab not found: ID does not exist" Jul 08 09:33:37.268941 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:37.268901 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh"] Jul 08 09:33:37.274616 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:37.274591 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-2b37457f-5bfd45ff65-2nznh"] Jul 08 09:33:38.308663 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:38.308578 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46773e2b-90b5-4b03-a9e0-130823581f65" path="/var/lib/kubelet/pods/46773e2b-90b5-4b03-a9e0-130823581f65/volumes" Jul 08 09:33:42.470570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.470546 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:42.632807 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.632693 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-socket\") pod \"13048f76-e2ff-4da9-b4fd-438c1b188a97\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " Jul 08 09:33:42.632807 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.632735 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-data\") pod \"13048f76-e2ff-4da9-b4fd-438c1b188a97\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " Jul 08 09:33:42.632807 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.632802 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prr9n\" (UniqueName: \"kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-kube-api-access-prr9n\") pod \"13048f76-e2ff-4da9-b4fd-438c1b188a97\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " Jul 08 09:33:42.633086 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.632847 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-credential-socket\") pod \"13048f76-e2ff-4da9-b4fd-438c1b188a97\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " Jul 08 09:33:42.633086 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.632872 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-envoy\") pod \"13048f76-e2ff-4da9-b4fd-438c1b188a97\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " Jul 08 09:33:42.633086 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.632898 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-token\") pod \"13048f76-e2ff-4da9-b4fd-438c1b188a97\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " Jul 08 09:33:42.633086 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.632944 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-podinfo\") pod \"13048f76-e2ff-4da9-b4fd-438c1b188a97\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " Jul 08 09:33:42.633086 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.632970 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13048f76-e2ff-4da9-b4fd-438c1b188a97-istiod-ca-cert\") pod \"13048f76-e2ff-4da9-b4fd-438c1b188a97\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " Jul 08 09:33:42.633362 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.633096 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "13048f76-e2ff-4da9-b4fd-438c1b188a97" (UID: "13048f76-e2ff-4da9-b4fd-438c1b188a97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:42.633362 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.633102 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-data" (OuterVolumeSpecName: "istio-data") pod "13048f76-e2ff-4da9-b4fd-438c1b188a97" (UID: "13048f76-e2ff-4da9-b4fd-438c1b188a97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:42.633362 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.633116 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "13048f76-e2ff-4da9-b4fd-438c1b188a97" (UID: "13048f76-e2ff-4da9-b4fd-438c1b188a97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:42.633362 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.633222 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-certs\") pod \"13048f76-e2ff-4da9-b4fd-438c1b188a97\" (UID: \"13048f76-e2ff-4da9-b4fd-438c1b188a97\") " Jul 08 09:33:42.633563 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.633372 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13048f76-e2ff-4da9-b4fd-438c1b188a97-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "13048f76-e2ff-4da9-b4fd-438c1b188a97" (UID: "13048f76-e2ff-4da9-b4fd-438c1b188a97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:33:42.633563 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.633520 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:42.633563 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.633540 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-data\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:42.633563 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.633557 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-credential-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:42.633717 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.633571 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13048f76-e2ff-4da9-b4fd-438c1b188a97-istiod-ca-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:42.633717 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.633615 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "13048f76-e2ff-4da9-b4fd-438c1b188a97" (UID: "13048f76-e2ff-4da9-b4fd-438c1b188a97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:42.635501 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.635470 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "13048f76-e2ff-4da9-b4fd-438c1b188a97" (UID: "13048f76-e2ff-4da9-b4fd-438c1b188a97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:42.635610 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.635538 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-token" (OuterVolumeSpecName: "istio-token") pod "13048f76-e2ff-4da9-b4fd-438c1b188a97" (UID: "13048f76-e2ff-4da9-b4fd-438c1b188a97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:33:42.635610 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.635598 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "13048f76-e2ff-4da9-b4fd-438c1b188a97" (UID: "13048f76-e2ff-4da9-b4fd-438c1b188a97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:33:42.635682 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.635610 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-kube-api-access-prr9n" (OuterVolumeSpecName: "kube-api-access-prr9n") pod "13048f76-e2ff-4da9-b4fd-438c1b188a97" (UID: "13048f76-e2ff-4da9-b4fd-438c1b188a97"). InnerVolumeSpecName "kube-api-access-prr9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:33:42.734637 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.734601 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-podinfo\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:42.734637 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.734628 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-workload-certs\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:42.734637 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.734640 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prr9n\" (UniqueName: \"kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-kube-api-access-prr9n\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:42.734884 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.734652 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-envoy\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:42.734884 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:42.734663 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13048f76-e2ff-4da9-b4fd-438c1b188a97-istio-token\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:33:43.255264 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:43.255230 2572 generic.go:356] "Generic (PLEG): container finished" podID="13048f76-e2ff-4da9-b4fd-438c1b188a97" containerID="0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6" exitCode=0 Jul 08 09:33:43.255451 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:43.255280 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" Jul 08 09:33:43.255451 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:43.255312 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" event={"ID":"13048f76-e2ff-4da9-b4fd-438c1b188a97","Type":"ContainerDied","Data":"0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6"} Jul 08 09:33:43.255451 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:43.255360 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft" event={"ID":"13048f76-e2ff-4da9-b4fd-438c1b188a97","Type":"ContainerDied","Data":"81aaceb6aad4c5fd633e16ceb506fd6176b04a992010cd1864252e8275941f04"} Jul 08 09:33:43.255451 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:43.255383 2572 scope.go:117] "RemoveContainer" containerID="0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6" Jul 08 09:33:43.264101 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:43.264081 2572 scope.go:117] "RemoveContainer" containerID="0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6" Jul 08 09:33:43.264343 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:33:43.264324 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6\": container with ID starting with 0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6 not found: ID does not exist" containerID="0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6" Jul 08 09:33:43.264405 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:43.264352 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6"} err="failed to get container status \"0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6\": rpc error: code = NotFound desc = could not find container \"0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6\": container with ID starting with 0600981ff40ba509921cef38e3339fe3fa7ab23382a697d68a231575c950a8d6 not found: ID does not exist" Jul 08 09:33:43.281500 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:43.281472 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft"] Jul 08 09:33:43.288397 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:43.288372 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2b37457f-openshift-default-6ff8b59c79-bw5ft"] Jul 08 09:33:44.310916 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:33:44.310877 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13048f76-e2ff-4da9-b4fd-438c1b188a97" path="/var/lib/kubelet/pods/13048f76-e2ff-4da9-b4fd-438c1b188a97/volumes" Jul 08 09:35:21.095602 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.095569 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc"] Jul 08 09:35:21.096070 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.095875 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46773e2b-90b5-4b03-a9e0-130823581f65" containerName="maas-api" Jul 08 09:35:21.096070 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.095887 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="46773e2b-90b5-4b03-a9e0-130823581f65" containerName="maas-api" Jul 08 09:35:21.096070 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.095899 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13048f76-e2ff-4da9-b4fd-438c1b188a97" containerName="istio-proxy" Jul 08 09:35:21.096070 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.095904 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="13048f76-e2ff-4da9-b4fd-438c1b188a97" containerName="istio-proxy" Jul 08 09:35:21.096070 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.095950 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="13048f76-e2ff-4da9-b4fd-438c1b188a97" containerName="istio-proxy" Jul 08 09:35:21.096070 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.095959 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="46773e2b-90b5-4b03-a9e0-130823581f65" containerName="maas-api" Jul 08 09:35:21.098860 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.098838 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.102231 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.102209 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1968caaf-openshift-default-dockercfg-nrmvn\"" Jul 08 09:35:21.154004 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.153972 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-certs\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.154185 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.154029 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld9js\" (UniqueName: \"kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-kube-api-access-ld9js\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.154185 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.154071 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-socket\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.154185 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.154099 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-podinfo\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.154185 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.154128 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-data\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.154185 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.154179 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-envoy\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.154368 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.154229 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istiod-ca-cert\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.154368 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.154263 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-credential-socket\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.154368 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.154290 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-token\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.239341 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.239303 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc"] Jul 08 09:35:21.255459 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.255429 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-certs\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.255630 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.255468 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld9js\" (UniqueName: \"kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-kube-api-access-ld9js\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.255630 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.255496 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-socket\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.255630 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.255513 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-podinfo\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.255630 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.255540 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-data\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.255630 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.255566 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-envoy\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.255916 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.255896 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-certs\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.255997 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.255963 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-socket\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.256119 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.256031 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istiod-ca-cert\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.256119 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.256091 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-credential-socket\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.256279 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.256118 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-token\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.256337 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.256271 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-data\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.256484 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.256460 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-credential-socket\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.256707 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.256683 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istiod-ca-cert\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.258280 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.258256 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-envoy\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.258644 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.258626 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-podinfo\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.275014 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.274992 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-token\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.276485 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.276460 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld9js\" (UniqueName: \"kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-kube-api-access-ld9js\") pod \"e2e-mt-1968caaf-openshift-default-5958775779-llvmc\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.408732 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.408635 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:21.583639 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.583609 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc"] Jul 08 09:35:21.585419 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:35:21.585394 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc924a63_89fe_44ff_8025_6cd0e3dafac2.slice/crio-1e5bf130db39fe101649f433c036bdb94ca6873b128ae18fc351aabee523f1f2 WatchSource:0}: Error finding container 1e5bf130db39fe101649f433c036bdb94ca6873b128ae18fc351aabee523f1f2: Status 404 returned error can't find the container with id 1e5bf130db39fe101649f433c036bdb94ca6873b128ae18fc351aabee523f1f2 Jul 08 09:35:21.587363 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.587335 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:35:21.587419 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.587398 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:35:21.587452 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.587425 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:35:21.695718 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.695630 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" event={"ID":"dc924a63-89fe-44ff-8025-6cd0e3dafac2","Type":"ContainerStarted","Data":"9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b"} Jul 08 09:35:21.695718 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.695687 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" event={"ID":"dc924a63-89fe-44ff-8025-6cd0e3dafac2","Type":"ContainerStarted","Data":"1e5bf130db39fe101649f433c036bdb94ca6873b128ae18fc351aabee523f1f2"} Jul 08 09:35:21.758198 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:21.758122 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" podStartSLOduration=0.758100675 podStartE2EDuration="758.100675ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:35:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:35:21.753841187 +0000 UTC m=+1720.003800469" watchObservedRunningTime="2026-07-08 09:35:21.758100675 +0000 UTC m=+1720.008059957" Jul 08 09:35:22.409304 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:22.409264 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:22.414199 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:22.414173 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:22.699821 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:22.699725 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:22.700709 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:22.700691 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:24.249152 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.249121 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb"] Jul 08 09:35:24.252575 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.252552 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:24.255174 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.255157 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-1968caaf-serving-cert\"" Jul 08 09:35:24.268516 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.268489 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb"] Jul 08 09:35:24.386894 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.386851 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h6lr\" (UniqueName: \"kubernetes.io/projected/6b477da0-034c-46a6-88f2-404b006c200d-kube-api-access-7h6lr\") pod \"maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:24.387125 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.387107 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b477da0-034c-46a6-88f2-404b006c200d-maas-api-tls\") pod \"maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:24.387284 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.387253 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b477da0-034c-46a6-88f2-404b006c200d-openshift-service-ca\") pod \"maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:24.488371 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.488340 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b477da0-034c-46a6-88f2-404b006c200d-maas-api-tls\") pod \"maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:24.488553 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.488395 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b477da0-034c-46a6-88f2-404b006c200d-openshift-service-ca\") pod \"maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:24.488553 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.488422 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h6lr\" (UniqueName: \"kubernetes.io/projected/6b477da0-034c-46a6-88f2-404b006c200d-kube-api-access-7h6lr\") pod \"maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:24.488553 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:35:24.488480 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-1968caaf-serving-cert: secret "maas-api-e2e-mt-1968caaf-serving-cert" not found Jul 08 09:35:24.488553 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:35:24.488554 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b477da0-034c-46a6-88f2-404b006c200d-maas-api-tls podName:6b477da0-034c-46a6-88f2-404b006c200d nodeName:}" failed. No retries permitted until 2026-07-08 09:35:24.988539157 +0000 UTC m=+1723.238498422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6b477da0-034c-46a6-88f2-404b006c200d-maas-api-tls") pod "maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" (UID: "6b477da0-034c-46a6-88f2-404b006c200d") : secret "maas-api-e2e-mt-1968caaf-serving-cert" not found Jul 08 09:35:24.489143 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.489117 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b477da0-034c-46a6-88f2-404b006c200d-openshift-service-ca\") pod \"maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:24.507295 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.507231 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h6lr\" (UniqueName: \"kubernetes.io/projected/6b477da0-034c-46a6-88f2-404b006c200d-kube-api-access-7h6lr\") pod \"maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:24.994161 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.994130 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b477da0-034c-46a6-88f2-404b006c200d-maas-api-tls\") pod \"maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:24.996580 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:24.996550 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b477da0-034c-46a6-88f2-404b006c200d-maas-api-tls\") pod \"maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:25.163272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:25.163226 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:25.309354 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:25.309329 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb"] Jul 08 09:35:25.311441 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:35:25.311411 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b477da0_034c_46a6_88f2_404b006c200d.slice/crio-0e0b9215907b253a73e4dc1611386b6944395ab227cba1ceb34332d620cc788c WatchSource:0}: Error finding container 0e0b9215907b253a73e4dc1611386b6944395ab227cba1ceb34332d620cc788c: Status 404 returned error can't find the container with id 0e0b9215907b253a73e4dc1611386b6944395ab227cba1ceb34332d620cc788c Jul 08 09:35:25.712803 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:25.712693 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" event={"ID":"6b477da0-034c-46a6-88f2-404b006c200d","Type":"ContainerStarted","Data":"0e0b9215907b253a73e4dc1611386b6944395ab227cba1ceb34332d620cc788c"} Jul 08 09:35:26.718424 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:26.718388 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" event={"ID":"6b477da0-034c-46a6-88f2-404b006c200d","Type":"ContainerStarted","Data":"2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6"} Jul 08 09:35:26.718892 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:26.718452 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:26.739694 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:26.739645 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" podStartSLOduration=2.318634921 podStartE2EDuration="2.739630138s" totalImagesPullingTime="420.995217ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:35:26.736940793 +0000 UTC m=+1724.986900084" watchObservedRunningTime="2026-07-08 09:35:26.739630138 +0000 UTC m=+1724.989589420" Jul 08 09:35:32.728028 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:32.728003 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:36.914856 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:36.914827 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb"] Jul 08 09:35:36.931881 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:36.915216 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" podUID="6b477da0-034c-46a6-88f2-404b006c200d" containerName="maas-api" containerID="cri-o://2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6" gracePeriod=30 Jul 08 09:35:37.153238 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.153212 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:37.304461 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.304424 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h6lr\" (UniqueName: \"kubernetes.io/projected/6b477da0-034c-46a6-88f2-404b006c200d-kube-api-access-7h6lr\") pod \"6b477da0-034c-46a6-88f2-404b006c200d\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " Jul 08 09:35:37.304631 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.304507 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b477da0-034c-46a6-88f2-404b006c200d-maas-api-tls\") pod \"6b477da0-034c-46a6-88f2-404b006c200d\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " Jul 08 09:35:37.304631 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.304584 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b477da0-034c-46a6-88f2-404b006c200d-openshift-service-ca\") pod \"6b477da0-034c-46a6-88f2-404b006c200d\" (UID: \"6b477da0-034c-46a6-88f2-404b006c200d\") " Jul 08 09:35:37.304922 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.304898 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b477da0-034c-46a6-88f2-404b006c200d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b477da0-034c-46a6-88f2-404b006c200d" (UID: "6b477da0-034c-46a6-88f2-404b006c200d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:35:37.306600 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.306571 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b477da0-034c-46a6-88f2-404b006c200d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b477da0-034c-46a6-88f2-404b006c200d" (UID: "6b477da0-034c-46a6-88f2-404b006c200d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:35:37.306702 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.306621 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b477da0-034c-46a6-88f2-404b006c200d-kube-api-access-7h6lr" (OuterVolumeSpecName: "kube-api-access-7h6lr") pod "6b477da0-034c-46a6-88f2-404b006c200d" (UID: "6b477da0-034c-46a6-88f2-404b006c200d"). InnerVolumeSpecName "kube-api-access-7h6lr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:35:37.406199 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.406168 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b477da0-034c-46a6-88f2-404b006c200d-openshift-service-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:37.406199 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.406201 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h6lr\" (UniqueName: \"kubernetes.io/projected/6b477da0-034c-46a6-88f2-404b006c200d-kube-api-access-7h6lr\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:37.406427 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.406216 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b477da0-034c-46a6-88f2-404b006c200d-maas-api-tls\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:37.701592 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.701503 2572 patch_prober.go:28] interesting pod/e2e-mt-1968caaf-openshift-default-5958775779-llvmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 08 09:35:37.701592 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.701564 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 08 09:35:37.770284 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.770256 2572 generic.go:356] "Generic (PLEG): container finished" podID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerID="9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b" exitCode=0 Jul 08 09:35:37.770488 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.770337 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" event={"ID":"dc924a63-89fe-44ff-8025-6cd0e3dafac2","Type":"ContainerDied","Data":"9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b"} Jul 08 09:35:37.770788 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.770746 2572 scope.go:117] "RemoveContainer" containerID="9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b" Jul 08 09:35:37.771507 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.771483 2572 generic.go:356] "Generic (PLEG): container finished" podID="6b477da0-034c-46a6-88f2-404b006c200d" containerID="2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6" exitCode=0 Jul 08 09:35:37.771613 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.771520 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" event={"ID":"6b477da0-034c-46a6-88f2-404b006c200d","Type":"ContainerDied","Data":"2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6"} Jul 08 09:35:37.771613 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.771531 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" Jul 08 09:35:37.771613 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.771558 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb" event={"ID":"6b477da0-034c-46a6-88f2-404b006c200d","Type":"ContainerDied","Data":"0e0b9215907b253a73e4dc1611386b6944395ab227cba1ceb34332d620cc788c"} Jul 08 09:35:37.771613 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.771587 2572 scope.go:117] "RemoveContainer" containerID="2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6" Jul 08 09:35:37.771833 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.771808 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:35:37.771900 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.771883 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:35:37.771957 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.771932 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:35:37.781324 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.781309 2572 scope.go:117] "RemoveContainer" containerID="2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6" Jul 08 09:35:37.781611 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:35:37.781595 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6\": container with ID starting with 2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6 not found: ID does not exist" containerID="2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6" Jul 08 09:35:37.781660 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.781619 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6"} err="failed to get container status \"2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6\": rpc error: code = NotFound desc = could not find container \"2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6\": container with ID starting with 2d337ee88b8f277d916f71c5d0aca5ac93bb6dc9f1af6738f65082f782048cd6 not found: ID does not exist" Jul 08 09:35:37.821089 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.811695 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb"] Jul 08 09:35:37.821089 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:37.816585 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-1968caaf-6445f88bbb-l5vrb"] Jul 08 09:35:38.307347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:38.307313 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b477da0-034c-46a6-88f2-404b006c200d" path="/var/lib/kubelet/pods/6b477da0-034c-46a6-88f2-404b006c200d/volumes" Jul 08 09:35:38.778823 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:38.778791 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" event={"ID":"dc924a63-89fe-44ff-8025-6cd0e3dafac2","Type":"ContainerStarted","Data":"38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c"} Jul 08 09:35:39.408859 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:39.408819 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:39.409279 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:39.409075 2572 patch_prober.go:28] interesting pod/e2e-mt-1968caaf-openshift-default-5958775779-llvmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 08 09:35:39.409279 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:39.409131 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 08 09:35:40.409850 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:40.409816 2572 patch_prober.go:28] interesting pod/e2e-mt-1968caaf-openshift-default-5958775779-llvmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 08 09:35:40.410230 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:40.409872 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 08 09:35:41.409309 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:41.409268 2572 patch_prober.go:28] interesting pod/e2e-mt-1968caaf-openshift-default-5958775779-llvmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 08 09:35:41.409508 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:41.409345 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 08 09:35:42.409516 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:42.409484 2572 patch_prober.go:28] interesting pod/e2e-mt-1968caaf-openshift-default-5958775779-llvmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 08 09:35:42.409887 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:42.409537 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 08 09:35:43.412973 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:43.412891 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:43.413398 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:43.413220 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:43.413971 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:43.413950 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:43.774100 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:43.774066 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc"] Jul 08 09:35:43.809955 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:43.809926 2572 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" secret="" err="secret \"e2e-mt-1968caaf-openshift-default-dockercfg-nrmvn\" not found" Jul 08 09:35:44.813961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:44.813926 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" containerID="cri-o://38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c" gracePeriod=30 Jul 08 09:35:50.068378 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.068355 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:50.106878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.106796 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld9js\" (UniqueName: \"kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-kube-api-access-ld9js\") pod \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " Jul 08 09:35:50.106878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.106840 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-socket\") pod \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " Jul 08 09:35:50.106878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.106880 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-data\") pod \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " Jul 08 09:35:50.107145 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.106921 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-certs\") pod \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " Jul 08 09:35:50.107145 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.106936 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-token\") pod \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " Jul 08 09:35:50.107145 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.106963 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-podinfo\") pod \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " Jul 08 09:35:50.107296 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.107226 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istiod-ca-cert\") pod \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " Jul 08 09:35:50.107296 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.107226 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc924a63-89fe-44ff-8025-6cd0e3dafac2" (UID: "dc924a63-89fe-44ff-8025-6cd0e3dafac2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:35:50.107404 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.107305 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-envoy\") pod \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " Jul 08 09:35:50.107404 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.107339 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-credential-socket\") pod \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\" (UID: \"dc924a63-89fe-44ff-8025-6cd0e3dafac2\") " Jul 08 09:35:50.107404 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.107375 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc924a63-89fe-44ff-8025-6cd0e3dafac2" (UID: "dc924a63-89fe-44ff-8025-6cd0e3dafac2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:35:50.107589 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.107564 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc924a63-89fe-44ff-8025-6cd0e3dafac2" (UID: "dc924a63-89fe-44ff-8025-6cd0e3dafac2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:35:50.107659 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.107635 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:50.107659 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.107660 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istiod-ca-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:50.107871 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.107728 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc924a63-89fe-44ff-8025-6cd0e3dafac2" (UID: "dc924a63-89fe-44ff-8025-6cd0e3dafac2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:35:50.108173 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.108142 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc924a63-89fe-44ff-8025-6cd0e3dafac2" (UID: "dc924a63-89fe-44ff-8025-6cd0e3dafac2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:35:50.109182 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.109142 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-kube-api-access-ld9js" (OuterVolumeSpecName: "kube-api-access-ld9js") pod "dc924a63-89fe-44ff-8025-6cd0e3dafac2" (UID: "dc924a63-89fe-44ff-8025-6cd0e3dafac2"). InnerVolumeSpecName "kube-api-access-ld9js". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:35:50.109629 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.109607 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc924a63-89fe-44ff-8025-6cd0e3dafac2" (UID: "dc924a63-89fe-44ff-8025-6cd0e3dafac2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:35:50.109734 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.109712 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc924a63-89fe-44ff-8025-6cd0e3dafac2" (UID: "dc924a63-89fe-44ff-8025-6cd0e3dafac2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:35:50.109949 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.109927 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc924a63-89fe-44ff-8025-6cd0e3dafac2" (UID: "dc924a63-89fe-44ff-8025-6cd0e3dafac2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:35:50.208436 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.208397 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-workload-certs\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:50.208436 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.208427 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-token\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:50.208436 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.208436 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-podinfo\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:50.208436 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.208443 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-envoy\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:50.208702 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.208451 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-credential-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:50.208702 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.208461 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld9js\" (UniqueName: \"kubernetes.io/projected/dc924a63-89fe-44ff-8025-6cd0e3dafac2-kube-api-access-ld9js\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:50.208702 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.208470 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc924a63-89fe-44ff-8025-6cd0e3dafac2-istio-data\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:35:50.842666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.842633 2572 generic.go:356] "Generic (PLEG): container finished" podID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerID="38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c" exitCode=0 Jul 08 09:35:50.842891 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.842679 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" Jul 08 09:35:50.842891 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.842718 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" event={"ID":"dc924a63-89fe-44ff-8025-6cd0e3dafac2","Type":"ContainerDied","Data":"38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c"} Jul 08 09:35:50.842891 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.842802 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc" event={"ID":"dc924a63-89fe-44ff-8025-6cd0e3dafac2","Type":"ContainerDied","Data":"1e5bf130db39fe101649f433c036bdb94ca6873b128ae18fc351aabee523f1f2"} Jul 08 09:35:50.842891 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.842835 2572 scope.go:117] "RemoveContainer" containerID="38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c" Jul 08 09:35:50.851025 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.851005 2572 scope.go:117] "RemoveContainer" containerID="9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b" Jul 08 09:35:50.858472 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.858457 2572 scope.go:117] "RemoveContainer" containerID="38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c" Jul 08 09:35:50.858716 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:35:50.858701 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c\": container with ID starting with 38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c not found: ID does not exist" containerID="38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c" Jul 08 09:35:50.858774 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.858724 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c"} err="failed to get container status \"38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c\": rpc error: code = NotFound desc = could not find container \"38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c\": container with ID starting with 38a2ab90cdc445aa034c250c46762ee132fb4ded300aa9d37226c5406ee54a7c not found: ID does not exist" Jul 08 09:35:50.858774 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.858740 2572 scope.go:117] "RemoveContainer" containerID="9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b" Jul 08 09:35:50.858994 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:35:50.858975 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b\": container with ID starting with 9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b not found: ID does not exist" containerID="9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b" Jul 08 09:35:50.859055 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.859004 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b"} err="failed to get container status \"9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b\": rpc error: code = NotFound desc = could not find container \"9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b\": container with ID starting with 9d59cc3511f69b16fab067b7ee67612024473834fba1e75d37249ef9afc20e7b not found: ID does not exist" Jul 08 09:35:50.868292 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.868270 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc"] Jul 08 09:35:50.877405 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:50.877374 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1968caaf-openshift-default-5958775779-llvmc"] Jul 08 09:35:52.307927 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:35:52.307887 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" path="/var/lib/kubelet/pods/dc924a63-89fe-44ff-8025-6cd0e3dafac2/volumes" Jul 08 09:36:04.179492 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.179456 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788"] Jul 08 09:36:04.179868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.179739 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" Jul 08 09:36:04.179868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.179764 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" Jul 08 09:36:04.179868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.179783 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b477da0-034c-46a6-88f2-404b006c200d" containerName="maas-api" Jul 08 09:36:04.179868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.179789 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b477da0-034c-46a6-88f2-404b006c200d" containerName="maas-api" Jul 08 09:36:04.179868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.179802 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" Jul 08 09:36:04.179868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.179807 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" Jul 08 09:36:04.179868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.179854 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b477da0-034c-46a6-88f2-404b006c200d" containerName="maas-api" Jul 08 09:36:04.179868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.179863 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" Jul 08 09:36:04.179868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.179868 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc924a63-89fe-44ff-8025-6cd0e3dafac2" containerName="istio-proxy" Jul 08 09:36:04.184071 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.184054 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.186858 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.186836 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0775092a-openshift-default-dockercfg-qghxh\"" Jul 08 09:36:04.200067 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.200043 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788"] Jul 08 09:36:04.233174 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.233134 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-socket\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.233174 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.233163 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-certs\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.233364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.233197 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-data\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.233364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.233224 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-envoy\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.233364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.233243 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a556684c-0162-4c15-b446-fbda8b5d468a-istio-podinfo\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.233364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.233268 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-istio-token\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.233364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.233336 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-credential-socket\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.233364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.233360 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swf8x\" (UniqueName: \"kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-kube-api-access-swf8x\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.233545 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.233382 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a556684c-0162-4c15-b446-fbda8b5d468a-istiod-ca-cert\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.334647 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.334619 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-socket\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.334647 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.334651 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-certs\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.334864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.334673 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-data\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.334864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.334690 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-envoy\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.334864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.334707 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a556684c-0162-4c15-b446-fbda8b5d468a-istio-podinfo\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.334864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.334761 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-istio-token\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.334864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.334860 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-credential-socket\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.335078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.334891 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swf8x\" (UniqueName: \"kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-kube-api-access-swf8x\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.335078 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.334923 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a556684c-0162-4c15-b446-fbda8b5d468a-istiod-ca-cert\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.335178 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.335073 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-certs\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.335178 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.335102 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-socket\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.335278 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.335213 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-data\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.335512 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.335485 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-credential-socket\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.335647 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.335629 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a556684c-0162-4c15-b446-fbda8b5d468a-istiod-ca-cert\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.337464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.337445 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-envoy\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.337799 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.337776 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a556684c-0162-4c15-b446-fbda8b5d468a-istio-podinfo\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.344014 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.343989 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-istio-token\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.344167 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.344149 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swf8x\" (UniqueName: \"kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-kube-api-access-swf8x\") pod \"e2e-mt-0775092a-openshift-default-fd64cd795-4h788\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.493961 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.493931 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:04.625963 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.625939 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788"] Jul 08 09:36:04.627904 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:36:04.627878 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda556684c_0162_4c15_b446_fbda8b5d468a.slice/crio-ef1da8fc00a8beda47eeecf040b8a10444b40b4ecd621fabdc3c034651cae9da WatchSource:0}: Error finding container ef1da8fc00a8beda47eeecf040b8a10444b40b4ecd621fabdc3c034651cae9da: Status 404 returned error can't find the container with id ef1da8fc00a8beda47eeecf040b8a10444b40b4ecd621fabdc3c034651cae9da Jul 08 09:36:04.630173 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.630141 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:36:04.630240 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.630209 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:36:04.630240 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.630236 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:36:04.908496 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.908398 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" event={"ID":"a556684c-0162-4c15-b446-fbda8b5d468a","Type":"ContainerStarted","Data":"c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c"} Jul 08 09:36:04.908496 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.908462 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" event={"ID":"a556684c-0162-4c15-b446-fbda8b5d468a","Type":"ContainerStarted","Data":"ef1da8fc00a8beda47eeecf040b8a10444b40b4ecd621fabdc3c034651cae9da"} Jul 08 09:36:04.938128 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:04.938071 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" podStartSLOduration=0.938054324 podStartE2EDuration="938.054324ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:36:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:36:04.935686248 +0000 UTC m=+1763.185645529" watchObservedRunningTime="2026-07-08 09:36:04.938054324 +0000 UTC m=+1763.188013606" Jul 08 09:36:05.494479 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:05.494441 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:05.499219 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:05.499197 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:05.914239 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:05.914145 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:05.915204 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:05.915181 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:07.125999 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.125964 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs"] Jul 08 09:36:07.129588 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.129566 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.133200 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.133181 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-0775092a-serving-cert\"" Jul 08 09:36:07.155901 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.155878 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs"] Jul 08 09:36:07.264209 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.264171 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce97ff35-8ca5-414d-96e9-2fa96659b842-openshift-service-ca\") pod \"maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.264356 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.264216 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97ff35-8ca5-414d-96e9-2fa96659b842-maas-api-tls\") pod \"maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.264356 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.264314 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxqsm\" (UniqueName: \"kubernetes.io/projected/ce97ff35-8ca5-414d-96e9-2fa96659b842-kube-api-access-vxqsm\") pod \"maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.365157 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.365117 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce97ff35-8ca5-414d-96e9-2fa96659b842-openshift-service-ca\") pod \"maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.365371 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.365162 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97ff35-8ca5-414d-96e9-2fa96659b842-maas-api-tls\") pod \"maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.365371 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.365232 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxqsm\" (UniqueName: \"kubernetes.io/projected/ce97ff35-8ca5-414d-96e9-2fa96659b842-kube-api-access-vxqsm\") pod \"maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.365774 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.365727 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce97ff35-8ca5-414d-96e9-2fa96659b842-openshift-service-ca\") pod \"maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.367526 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.367506 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97ff35-8ca5-414d-96e9-2fa96659b842-maas-api-tls\") pod \"maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.374417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.374396 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxqsm\" (UniqueName: \"kubernetes.io/projected/ce97ff35-8ca5-414d-96e9-2fa96659b842-kube-api-access-vxqsm\") pod \"maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.439439 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.439347 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:07.582413 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.582390 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs"] Jul 08 09:36:07.583919 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:36:07.583894 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce97ff35_8ca5_414d_96e9_2fa96659b842.slice/crio-fa5b506740110e2e1cf23ee9979baaf8e7a86e951b1d1f4b3087bf02831e0896 WatchSource:0}: Error finding container fa5b506740110e2e1cf23ee9979baaf8e7a86e951b1d1f4b3087bf02831e0896: Status 404 returned error can't find the container with id fa5b506740110e2e1cf23ee9979baaf8e7a86e951b1d1f4b3087bf02831e0896 Jul 08 09:36:07.934924 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:07.934892 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" event={"ID":"ce97ff35-8ca5-414d-96e9-2fa96659b842","Type":"ContainerStarted","Data":"fa5b506740110e2e1cf23ee9979baaf8e7a86e951b1d1f4b3087bf02831e0896"} Jul 08 09:36:08.940422 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:08.940382 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" event={"ID":"ce97ff35-8ca5-414d-96e9-2fa96659b842","Type":"ContainerStarted","Data":"655eb74c8a76cb852114288422dc95af333c97b641dded4085e95dabec6f4e8c"} Jul 08 09:36:08.940861 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:08.940534 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:08.963467 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:08.963416 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" podStartSLOduration=1.534600126 podStartE2EDuration="1.963399392s" totalImagesPullingTime="428.799266ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:36:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:36:08.960895528 +0000 UTC m=+1767.210854812" watchObservedRunningTime="2026-07-08 09:36:08.963399392 +0000 UTC m=+1767.213358707" Jul 08 09:36:14.951319 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:14.951234 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:17.984933 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:17.984902 2572 generic.go:356] "Generic (PLEG): container finished" podID="a556684c-0162-4c15-b446-fbda8b5d468a" containerID="c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c" exitCode=0 Jul 08 09:36:17.985360 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:17.984972 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" event={"ID":"a556684c-0162-4c15-b446-fbda8b5d468a","Type":"ContainerDied","Data":"c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c"} Jul 08 09:36:17.985415 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:17.985398 2572 scope.go:117] "RemoveContainer" containerID="c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c" Jul 08 09:36:17.986501 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:17.986468 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:36:17.986570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:17.986535 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:36:17.986606 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:17.986567 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:36:19.009091 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:19.009039 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" event={"ID":"a556684c-0162-4c15-b446-fbda8b5d468a","Type":"ContainerStarted","Data":"777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b"} Jul 08 09:36:19.496158 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:19.496126 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:19.496543 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:19.496517 2572 patch_prober.go:28] interesting pod/e2e-mt-0775092a-openshift-default-fd64cd795-4h788 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 09:36:19.496713 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:19.496690 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 09:36:20.495055 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:20.495010 2572 patch_prober.go:28] interesting pod/e2e-mt-0775092a-openshift-default-fd64cd795-4h788 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 09:36:20.495556 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:20.495076 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 09:36:20.915162 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:20.915070 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:21.495780 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:21.494818 2572 patch_prober.go:28] interesting pod/e2e-mt-0775092a-openshift-default-fd64cd795-4h788 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 09:36:21.495780 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:21.494879 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 09:36:22.494452 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:22.494408 2572 patch_prober.go:28] interesting pod/e2e-mt-0775092a-openshift-default-fd64cd795-4h788 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 09:36:22.494839 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:22.494804 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 09:36:23.498053 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:23.498025 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:23.499117 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:23.499097 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:36:54.985911 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:54.985875 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs"] Jul 08 09:36:54.986388 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:54.986108 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" podUID="ce97ff35-8ca5-414d-96e9-2fa96659b842" containerName="maas-api" containerID="cri-o://655eb74c8a76cb852114288422dc95af333c97b641dded4085e95dabec6f4e8c" gracePeriod=30 Jul 08 09:36:55.179704 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.179673 2572 generic.go:356] "Generic (PLEG): container finished" podID="ce97ff35-8ca5-414d-96e9-2fa96659b842" containerID="655eb74c8a76cb852114288422dc95af333c97b641dded4085e95dabec6f4e8c" exitCode=0 Jul 08 09:36:55.179876 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.179728 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" event={"ID":"ce97ff35-8ca5-414d-96e9-2fa96659b842","Type":"ContainerDied","Data":"655eb74c8a76cb852114288422dc95af333c97b641dded4085e95dabec6f4e8c"} Jul 08 09:36:55.223168 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.223145 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:55.286269 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.286185 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce97ff35-8ca5-414d-96e9-2fa96659b842-openshift-service-ca\") pod \"ce97ff35-8ca5-414d-96e9-2fa96659b842\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " Jul 08 09:36:55.286269 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.286218 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97ff35-8ca5-414d-96e9-2fa96659b842-maas-api-tls\") pod \"ce97ff35-8ca5-414d-96e9-2fa96659b842\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " Jul 08 09:36:55.286269 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.286235 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxqsm\" (UniqueName: \"kubernetes.io/projected/ce97ff35-8ca5-414d-96e9-2fa96659b842-kube-api-access-vxqsm\") pod \"ce97ff35-8ca5-414d-96e9-2fa96659b842\" (UID: \"ce97ff35-8ca5-414d-96e9-2fa96659b842\") " Jul 08 09:36:55.286570 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.286545 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce97ff35-8ca5-414d-96e9-2fa96659b842-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce97ff35-8ca5-414d-96e9-2fa96659b842" (UID: "ce97ff35-8ca5-414d-96e9-2fa96659b842"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:36:55.288300 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.288274 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce97ff35-8ca5-414d-96e9-2fa96659b842-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce97ff35-8ca5-414d-96e9-2fa96659b842" (UID: "ce97ff35-8ca5-414d-96e9-2fa96659b842"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:36:55.288406 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.288367 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce97ff35-8ca5-414d-96e9-2fa96659b842-kube-api-access-vxqsm" (OuterVolumeSpecName: "kube-api-access-vxqsm") pod "ce97ff35-8ca5-414d-96e9-2fa96659b842" (UID: "ce97ff35-8ca5-414d-96e9-2fa96659b842"). InnerVolumeSpecName "kube-api-access-vxqsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:36:55.387047 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.387012 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce97ff35-8ca5-414d-96e9-2fa96659b842-openshift-service-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:36:55.387047 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.387041 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97ff35-8ca5-414d-96e9-2fa96659b842-maas-api-tls\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:36:55.387047 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:55.387050 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxqsm\" (UniqueName: \"kubernetes.io/projected/ce97ff35-8ca5-414d-96e9-2fa96659b842-kube-api-access-vxqsm\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:36:56.186873 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:56.186833 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" event={"ID":"ce97ff35-8ca5-414d-96e9-2fa96659b842","Type":"ContainerDied","Data":"fa5b506740110e2e1cf23ee9979baaf8e7a86e951b1d1f4b3087bf02831e0896"} Jul 08 09:36:56.186873 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:56.186871 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs" Jul 08 09:36:56.187364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:56.186906 2572 scope.go:117] "RemoveContainer" containerID="655eb74c8a76cb852114288422dc95af333c97b641dded4085e95dabec6f4e8c" Jul 08 09:36:56.224430 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:56.224389 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs"] Jul 08 09:36:56.227506 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:56.227483 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-0775092a-57f64bdb9d-zmxvs"] Jul 08 09:36:56.308375 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:36:56.308343 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce97ff35-8ca5-414d-96e9-2fa96659b842" path="/var/lib/kubelet/pods/ce97ff35-8ca5-414d-96e9-2fa96659b842/volumes" Jul 08 09:37:49.419537 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:49.419496 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788"] Jul 08 09:37:49.420070 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:49.419714 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" containerID="cri-o://777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b" gracePeriod=30 Jul 08 09:37:53.501038 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:53.501000 2572 patch_prober.go:28] interesting pod/e2e-mt-0775092a-openshift-default-fd64cd795-4h788 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 09:37:53.501422 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:53.501093 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 09:37:54.659467 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.659439 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:37:54.776157 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776124 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-data\") pod \"a556684c-0162-4c15-b446-fbda8b5d468a\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " Jul 08 09:37:54.776364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776167 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swf8x\" (UniqueName: \"kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-kube-api-access-swf8x\") pod \"a556684c-0162-4c15-b446-fbda8b5d468a\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " Jul 08 09:37:54.776364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776201 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a556684c-0162-4c15-b446-fbda8b5d468a-istiod-ca-cert\") pod \"a556684c-0162-4c15-b446-fbda8b5d468a\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " Jul 08 09:37:54.776364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776226 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-istio-token\") pod \"a556684c-0162-4c15-b446-fbda8b5d468a\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " Jul 08 09:37:54.776364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776266 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a556684c-0162-4c15-b446-fbda8b5d468a-istio-podinfo\") pod \"a556684c-0162-4c15-b446-fbda8b5d468a\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " Jul 08 09:37:54.776364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776314 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-credential-socket\") pod \"a556684c-0162-4c15-b446-fbda8b5d468a\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " Jul 08 09:37:54.776364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776355 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-socket\") pod \"a556684c-0162-4c15-b446-fbda8b5d468a\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " Jul 08 09:37:54.776661 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776388 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-envoy\") pod \"a556684c-0162-4c15-b446-fbda8b5d468a\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " Jul 08 09:37:54.776661 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776431 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-certs\") pod \"a556684c-0162-4c15-b446-fbda8b5d468a\" (UID: \"a556684c-0162-4c15-b446-fbda8b5d468a\") " Jul 08 09:37:54.776661 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776636 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a556684c-0162-4c15-b446-fbda8b5d468a" (UID: "a556684c-0162-4c15-b446-fbda8b5d468a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:37:54.776845 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776735 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a556684c-0162-4c15-b446-fbda8b5d468a" (UID: "a556684c-0162-4c15-b446-fbda8b5d468a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:37:54.776845 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776671 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a556684c-0162-4c15-b446-fbda8b5d468a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a556684c-0162-4c15-b446-fbda8b5d468a" (UID: "a556684c-0162-4c15-b446-fbda8b5d468a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:37:54.776954 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.776902 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a556684c-0162-4c15-b446-fbda8b5d468a" (UID: "a556684c-0162-4c15-b446-fbda8b5d468a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:37:54.777127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.777106 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-data" (OuterVolumeSpecName: "istio-data") pod "a556684c-0162-4c15-b446-fbda8b5d468a" (UID: "a556684c-0162-4c15-b446-fbda8b5d468a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:37:54.778613 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.778589 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-istio-token" (OuterVolumeSpecName: "istio-token") pod "a556684c-0162-4c15-b446-fbda8b5d468a" (UID: "a556684c-0162-4c15-b446-fbda8b5d468a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:37:54.778739 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.778638 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a556684c-0162-4c15-b446-fbda8b5d468a" (UID: "a556684c-0162-4c15-b446-fbda8b5d468a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:37:54.778886 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.778814 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-kube-api-access-swf8x" (OuterVolumeSpecName: "kube-api-access-swf8x") pod "a556684c-0162-4c15-b446-fbda8b5d468a" (UID: "a556684c-0162-4c15-b446-fbda8b5d468a"). InnerVolumeSpecName "kube-api-access-swf8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:37:54.778886 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.778820 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a556684c-0162-4c15-b446-fbda8b5d468a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a556684c-0162-4c15-b446-fbda8b5d468a" (UID: "a556684c-0162-4c15-b446-fbda8b5d468a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:37:54.877958 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.877927 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-credential-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:37:54.877958 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.877953 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:37:54.877958 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.877962 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-envoy\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:37:54.877958 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.877970 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-workload-certs\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:37:54.878314 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.877977 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a556684c-0162-4c15-b446-fbda8b5d468a-istio-data\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:37:54.878314 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.877985 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swf8x\" (UniqueName: \"kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-kube-api-access-swf8x\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:37:54.878314 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.877993 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a556684c-0162-4c15-b446-fbda8b5d468a-istiod-ca-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:37:54.878314 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.878001 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a556684c-0162-4c15-b446-fbda8b5d468a-istio-token\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:37:54.878314 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:54.878009 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a556684c-0162-4c15-b446-fbda8b5d468a-istio-podinfo\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:37:55.455727 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.455691 2572 generic.go:356] "Generic (PLEG): container finished" podID="a556684c-0162-4c15-b446-fbda8b5d468a" containerID="777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b" exitCode=0 Jul 08 09:37:55.455932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.455737 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" Jul 08 09:37:55.455932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.455770 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" event={"ID":"a556684c-0162-4c15-b446-fbda8b5d468a","Type":"ContainerDied","Data":"777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b"} Jul 08 09:37:55.455932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.455831 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788" event={"ID":"a556684c-0162-4c15-b446-fbda8b5d468a","Type":"ContainerDied","Data":"ef1da8fc00a8beda47eeecf040b8a10444b40b4ecd621fabdc3c034651cae9da"} Jul 08 09:37:55.455932 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.455856 2572 scope.go:117] "RemoveContainer" containerID="777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b" Jul 08 09:37:55.465046 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.465030 2572 scope.go:117] "RemoveContainer" containerID="c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c" Jul 08 09:37:55.472585 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.472568 2572 scope.go:117] "RemoveContainer" containerID="777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b" Jul 08 09:37:55.472860 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:37:55.472833 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b\": container with ID starting with 777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b not found: ID does not exist" containerID="777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b" Jul 08 09:37:55.472933 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.472872 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b"} err="failed to get container status \"777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b\": rpc error: code = NotFound desc = could not find container \"777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b\": container with ID starting with 777f5bf841edb63288d00c623bf0d526a30a52ef0b131edee3352868d33c3d9b not found: ID does not exist" Jul 08 09:37:55.472933 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.472889 2572 scope.go:117] "RemoveContainer" containerID="c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c" Jul 08 09:37:55.473115 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:37:55.473102 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c\": container with ID starting with c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c not found: ID does not exist" containerID="c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c" Jul 08 09:37:55.473150 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.473120 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c"} err="failed to get container status \"c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c\": rpc error: code = NotFound desc = could not find container \"c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c\": container with ID starting with c4b24e3e01e95b700ce49367e2897d1e720a534ca6c0f3a4b179ec65eed08b9c not found: ID does not exist" Jul 08 09:37:55.480005 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.479983 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788"] Jul 08 09:37:55.492939 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:55.491476 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0775092a-openshift-default-fd64cd795-4h788"] Jul 08 09:37:56.307956 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:37:56.307927 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" path="/var/lib/kubelet/pods/a556684c-0162-4c15-b446-fbda8b5d468a/volumes" Jul 08 09:38:45.376796 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.376768 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4"] Jul 08 09:38:45.377197 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.377062 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce97ff35-8ca5-414d-96e9-2fa96659b842" containerName="maas-api" Jul 08 09:38:45.377197 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.377077 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce97ff35-8ca5-414d-96e9-2fa96659b842" containerName="maas-api" Jul 08 09:38:45.377197 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.377088 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" Jul 08 09:38:45.377197 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.377094 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" Jul 08 09:38:45.377197 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.377110 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" Jul 08 09:38:45.377197 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.377117 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" Jul 08 09:38:45.377197 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.377172 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce97ff35-8ca5-414d-96e9-2fa96659b842" containerName="maas-api" Jul 08 09:38:45.377197 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.377179 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" Jul 08 09:38:45.377197 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.377195 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a556684c-0162-4c15-b446-fbda8b5d468a" containerName="istio-proxy" Jul 08 09:38:45.381260 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.381240 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.384177 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.384157 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2ec6b4-openshift-default-dockercfg-rxxrg\"" Jul 08 09:38:45.396270 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.396248 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4"] Jul 08 09:38:45.499526 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.499490 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-credential-socket\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.499695 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.499529 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zdz5\" (UniqueName: \"kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-kube-api-access-8zdz5\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.499695 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.499617 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-podinfo\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.499809 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.499718 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-data\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.499809 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.499743 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-envoy\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.499809 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.499784 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-token\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.499809 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.499805 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-certs\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.499935 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.499834 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-socket\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.499935 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.499872 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istiod-ca-cert\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.600464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.600431 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-credential-socket\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.600464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.600471 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zdz5\" (UniqueName: \"kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-kube-api-access-8zdz5\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.600733 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.600500 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-podinfo\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.600733 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.600546 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-data\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.600733 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.600568 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-envoy\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.600733 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.600583 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-token\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.600733 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.600612 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-certs\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.600733 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.600654 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-socket\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.600733 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.600677 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istiod-ca-cert\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.601096 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.600911 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-credential-socket\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.601096 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.601036 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-data\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.601180 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.601087 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-certs\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.601241 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.601227 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-socket\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.601375 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.601359 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istiod-ca-cert\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.603360 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.603339 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-envoy\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.603690 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.603669 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-podinfo\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.610192 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.610170 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-token\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.610549 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.610529 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zdz5\" (UniqueName: \"kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-kube-api-access-8zdz5\") pod \"e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.691370 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.691288 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:45.826253 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.826215 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4"] Jul 08 09:38:45.827782 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:38:45.827736 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3712744_d17c_4b51_b5ac_f5c95d67e08b.slice/crio-10176cf1b5de8adb6029b7ef4e6cd489cbd10969db78094a7f92e4e21d28ad90 WatchSource:0}: Error finding container 10176cf1b5de8adb6029b7ef4e6cd489cbd10969db78094a7f92e4e21d28ad90: Status 404 returned error can't find the container with id 10176cf1b5de8adb6029b7ef4e6cd489cbd10969db78094a7f92e4e21d28ad90 Jul 08 09:38:45.829817 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.829784 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:38:45.829901 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.829845 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:38:45.829901 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:45.829880 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:38:46.688633 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:46.688596 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" event={"ID":"b3712744-d17c-4b51-b5ac-f5c95d67e08b","Type":"ContainerStarted","Data":"7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03"} Jul 08 09:38:46.689025 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:46.688677 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" event={"ID":"b3712744-d17c-4b51-b5ac-f5c95d67e08b","Type":"ContainerStarted","Data":"10176cf1b5de8adb6029b7ef4e6cd489cbd10969db78094a7f92e4e21d28ad90"} Jul 08 09:38:46.692061 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:46.692030 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:46.697116 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:46.697092 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:46.714332 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:46.714277 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" podStartSLOduration=1.71426235 podStartE2EDuration="1.71426235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:38:46.711958733 +0000 UTC m=+1924.961918016" watchObservedRunningTime="2026-07-08 09:38:46.71426235 +0000 UTC m=+1924.964221711" Jul 08 09:38:47.693976 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:47.693948 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:47.695040 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:47.695019 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:38:48.851339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:48.851310 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z"] Jul 08 09:38:48.854608 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:48.854591 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:48.859008 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:48.858988 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-2ec6b4-serving-cert\"" Jul 08 09:38:48.881597 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:48.881572 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z"] Jul 08 09:38:48.935265 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:48.935224 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr97x\" (UniqueName: \"kubernetes.io/projected/41e3b748-9ddd-43dc-93ec-05144988f6c9-kube-api-access-fr97x\") pod \"maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:48.935468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:48.935276 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41e3b748-9ddd-43dc-93ec-05144988f6c9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:48.935468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:48.935370 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e3b748-9ddd-43dc-93ec-05144988f6c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:49.036349 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:49.036311 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr97x\" (UniqueName: \"kubernetes.io/projected/41e3b748-9ddd-43dc-93ec-05144988f6c9-kube-api-access-fr97x\") pod \"maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:49.036523 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:49.036358 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41e3b748-9ddd-43dc-93ec-05144988f6c9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:49.036523 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:49.036430 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e3b748-9ddd-43dc-93ec-05144988f6c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:49.036600 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:38:49.036569 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-2ec6b4-serving-cert: secret "maas-api-e2e-inf-a-2ec6b4-serving-cert" not found Jul 08 09:38:49.036659 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:38:49.036646 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e3b748-9ddd-43dc-93ec-05144988f6c9-maas-api-tls podName:41e3b748-9ddd-43dc-93ec-05144988f6c9 nodeName:}" failed. No retries permitted until 2026-07-08 09:38:49.536624578 +0000 UTC m=+1927.786583839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/41e3b748-9ddd-43dc-93ec-05144988f6c9-maas-api-tls") pod "maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" (UID: "41e3b748-9ddd-43dc-93ec-05144988f6c9") : secret "maas-api-e2e-inf-a-2ec6b4-serving-cert" not found Jul 08 09:38:49.037252 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:49.037229 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41e3b748-9ddd-43dc-93ec-05144988f6c9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:49.051746 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:49.051720 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr97x\" (UniqueName: \"kubernetes.io/projected/41e3b748-9ddd-43dc-93ec-05144988f6c9-kube-api-access-fr97x\") pod \"maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:49.540343 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:49.540306 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e3b748-9ddd-43dc-93ec-05144988f6c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:49.542878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:49.542854 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e3b748-9ddd-43dc-93ec-05144988f6c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:49.765481 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:49.765447 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:49.892837 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:49.892812 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z"] Jul 08 09:38:49.894849 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:38:49.894819 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41e3b748_9ddd_43dc_93ec_05144988f6c9.slice/crio-90a04c420b5ae3c832cba2223409581eb25d739b786495f4d863c2d727af4b7a WatchSource:0}: Error finding container 90a04c420b5ae3c832cba2223409581eb25d739b786495f4d863c2d727af4b7a: Status 404 returned error can't find the container with id 90a04c420b5ae3c832cba2223409581eb25d739b786495f4d863c2d727af4b7a Jul 08 09:38:49.896290 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:49.896273 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 09:38:50.712386 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:50.712349 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" event={"ID":"41e3b748-9ddd-43dc-93ec-05144988f6c9","Type":"ContainerStarted","Data":"fd0e9e53b405d7a4c122ba1de9eafd4e4b14a6c094414eacf5d73f4851093776"} Jul 08 09:38:50.712582 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:50.712400 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" event={"ID":"41e3b748-9ddd-43dc-93ec-05144988f6c9","Type":"ContainerStarted","Data":"90a04c420b5ae3c832cba2223409581eb25d739b786495f4d863c2d727af4b7a"} Jul 08 09:38:50.712582 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:50.712468 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:50.733718 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:50.733666 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" podStartSLOduration=2.07176829 podStartE2EDuration="2.733650207s" totalImagesPullingTime="661.881917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:38:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:38:50.73132045 +0000 UTC m=+1928.981279733" watchObservedRunningTime="2026-07-08 09:38:50.733650207 +0000 UTC m=+1928.983609489" Jul 08 09:38:56.721766 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:56.721721 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:38:57.435094 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.435059 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw"] Jul 08 09:38:57.439358 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.439340 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:57.443969 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.443944 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-802008-serving-cert\"" Jul 08 09:38:57.454370 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.454337 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw"] Jul 08 09:38:57.514509 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.514472 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsvgg\" (UniqueName: \"kubernetes.io/projected/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-kube-api-access-gsvgg\") pod \"maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:57.514676 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.514542 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-maas-api-tls\") pod \"maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:57.514676 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.514573 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:57.615936 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.615895 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-maas-api-tls\") pod \"maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:57.616149 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.615948 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:57.616149 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.615977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsvgg\" (UniqueName: \"kubernetes.io/projected/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-kube-api-access-gsvgg\") pod \"maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:57.616149 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:38:57.616046 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-802008-serving-cert: secret "maas-api-e2e-inf-b-802008-serving-cert" not found Jul 08 09:38:57.616149 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:38:57.616129 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-maas-api-tls podName:02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c nodeName:}" failed. No retries permitted until 2026-07-08 09:38:58.116111965 +0000 UTC m=+1936.366071230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-maas-api-tls") pod "maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" (UID: "02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c") : secret "maas-api-e2e-inf-b-802008-serving-cert" not found Jul 08 09:38:57.616674 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.616649 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:57.630258 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:57.630228 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsvgg\" (UniqueName: \"kubernetes.io/projected/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-kube-api-access-gsvgg\") pod \"maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:58.120159 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:58.120125 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-maas-api-tls\") pod \"maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:58.122515 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:58.122479 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-maas-api-tls\") pod \"maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:58.349649 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:58.349621 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:58.520790 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:58.520765 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw"] Jul 08 09:38:58.522280 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:38:58.522252 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c0ad5f_ca6f_40bc_a16d_d5e3abd06a7c.slice/crio-6e85f673ab28d180118ea251212ff7ed767f2145c195ed5bc756f557e46aaf34 WatchSource:0}: Error finding container 6e85f673ab28d180118ea251212ff7ed767f2145c195ed5bc756f557e46aaf34: Status 404 returned error can't find the container with id 6e85f673ab28d180118ea251212ff7ed767f2145c195ed5bc756f557e46aaf34 Jul 08 09:38:58.751300 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:58.751266 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" event={"ID":"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c","Type":"ContainerStarted","Data":"6e85f673ab28d180118ea251212ff7ed767f2145c195ed5bc756f557e46aaf34"} Jul 08 09:38:59.757722 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:59.757689 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" event={"ID":"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c","Type":"ContainerStarted","Data":"0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65"} Jul 08 09:38:59.758094 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:59.757866 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:38:59.778685 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:38:59.778616 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" podStartSLOduration=2.232515887 podStartE2EDuration="2.778594718s" totalImagesPullingTime="546.078831ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:38:59.777100215 +0000 UTC m=+1938.027059497" watchObservedRunningTime="2026-07-08 09:38:59.778594718 +0000 UTC m=+1938.028554000" Jul 08 09:39:03.358769 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:03.358703 2572 patch_prober.go:28] interesting pod/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:57266->10.132.0.42:15021: read: connection reset by peer" start-of-body= Jul 08 09:39:03.359148 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:03.358808 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:57266->10.132.0.42:15021: read: connection reset by peer" Jul 08 09:39:03.777532 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:03.777496 2572 generic.go:356] "Generic (PLEG): container finished" podID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerID="7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03" exitCode=0 Jul 08 09:39:03.777698 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:03.777572 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" event={"ID":"b3712744-d17c-4b51-b5ac-f5c95d67e08b","Type":"ContainerDied","Data":"7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03"} Jul 08 09:39:03.778010 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:03.777998 2572 scope.go:117] "RemoveContainer" containerID="7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03" Jul 08 09:39:03.778978 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:03.778950 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:39:03.779039 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:03.779017 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:39:03.779075 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:03.779048 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:39:04.787222 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:04.787176 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" event={"ID":"b3712744-d17c-4b51-b5ac-f5c95d67e08b","Type":"ContainerStarted","Data":"6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d"} Jul 08 09:39:05.693219 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:05.692776 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:39:05.693219 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:05.692903 2572 patch_prober.go:28] interesting pod/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 08 09:39:05.693219 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:05.692946 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 08 09:39:05.768912 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:05.768887 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:39:06.691873 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:06.691834 2572 patch_prober.go:28] interesting pod/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 08 09:39:06.692455 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:06.692425 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 08 09:39:07.692631 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:07.692585 2572 patch_prober.go:28] interesting pod/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 08 09:39:07.693162 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:07.692655 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 08 09:39:08.692162 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:08.692126 2572 patch_prober.go:28] interesting pod/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 08 09:39:08.692347 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:08.692184 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 08 09:39:09.695931 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:09.695903 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:39:09.696304 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:09.696221 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:39:09.696856 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:39:09.696839 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:40:12.880207 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:12.880175 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z"] Jul 08 09:40:12.880668 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:12.880382 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" podUID="41e3b748-9ddd-43dc-93ec-05144988f6c9" containerName="maas-api" containerID="cri-o://fd0e9e53b405d7a4c122ba1de9eafd4e4b14a6c094414eacf5d73f4851093776" gracePeriod=30 Jul 08 09:40:13.108652 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.108619 2572 generic.go:356] "Generic (PLEG): container finished" podID="41e3b748-9ddd-43dc-93ec-05144988f6c9" containerID="fd0e9e53b405d7a4c122ba1de9eafd4e4b14a6c094414eacf5d73f4851093776" exitCode=0 Jul 08 09:40:13.108807 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.108705 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" event={"ID":"41e3b748-9ddd-43dc-93ec-05144988f6c9","Type":"ContainerDied","Data":"fd0e9e53b405d7a4c122ba1de9eafd4e4b14a6c094414eacf5d73f4851093776"} Jul 08 09:40:13.108807 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.108801 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" event={"ID":"41e3b748-9ddd-43dc-93ec-05144988f6c9","Type":"ContainerDied","Data":"90a04c420b5ae3c832cba2223409581eb25d739b786495f4d863c2d727af4b7a"} Jul 08 09:40:13.108894 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.108829 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90a04c420b5ae3c832cba2223409581eb25d739b786495f4d863c2d727af4b7a" Jul 08 09:40:13.117966 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.117945 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:40:13.252953 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.252926 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41e3b748-9ddd-43dc-93ec-05144988f6c9-openshift-service-ca\") pod \"41e3b748-9ddd-43dc-93ec-05144988f6c9\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " Jul 08 09:40:13.253131 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.252987 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e3b748-9ddd-43dc-93ec-05144988f6c9-maas-api-tls\") pod \"41e3b748-9ddd-43dc-93ec-05144988f6c9\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " Jul 08 09:40:13.253131 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.253050 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr97x\" (UniqueName: \"kubernetes.io/projected/41e3b748-9ddd-43dc-93ec-05144988f6c9-kube-api-access-fr97x\") pod \"41e3b748-9ddd-43dc-93ec-05144988f6c9\" (UID: \"41e3b748-9ddd-43dc-93ec-05144988f6c9\") " Jul 08 09:40:13.253359 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.253331 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41e3b748-9ddd-43dc-93ec-05144988f6c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "41e3b748-9ddd-43dc-93ec-05144988f6c9" (UID: "41e3b748-9ddd-43dc-93ec-05144988f6c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:40:13.255072 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.255044 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41e3b748-9ddd-43dc-93ec-05144988f6c9-kube-api-access-fr97x" (OuterVolumeSpecName: "kube-api-access-fr97x") pod "41e3b748-9ddd-43dc-93ec-05144988f6c9" (UID: "41e3b748-9ddd-43dc-93ec-05144988f6c9"). InnerVolumeSpecName "kube-api-access-fr97x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:40:13.255072 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.255053 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41e3b748-9ddd-43dc-93ec-05144988f6c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "41e3b748-9ddd-43dc-93ec-05144988f6c9" (UID: "41e3b748-9ddd-43dc-93ec-05144988f6c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:40:13.353796 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.353746 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr97x\" (UniqueName: \"kubernetes.io/projected/41e3b748-9ddd-43dc-93ec-05144988f6c9-kube-api-access-fr97x\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:40:13.353796 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.353788 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41e3b748-9ddd-43dc-93ec-05144988f6c9-openshift-service-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:40:13.353796 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:13.353798 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41e3b748-9ddd-43dc-93ec-05144988f6c9-maas-api-tls\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:40:14.113142 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:14.113112 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z" Jul 08 09:40:14.136122 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:14.136094 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z"] Jul 08 09:40:14.140284 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:14.140258 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2ec6b4-5df8876d99-vwb2z"] Jul 08 09:40:14.307761 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:14.307728 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41e3b748-9ddd-43dc-93ec-05144988f6c9" path="/var/lib/kubelet/pods/41e3b748-9ddd-43dc-93ec-05144988f6c9/volumes" Jul 08 09:40:54.935983 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:54.935952 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4"] Jul 08 09:40:54.936518 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:54.936175 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" containerID="cri-o://6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d" gracePeriod=30 Jul 08 09:40:56.574991 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.574958 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw"] Jul 08 09:40:56.575369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.575164 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" podUID="02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c" containerName="maas-api" containerID="cri-o://0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65" gracePeriod=30 Jul 08 09:40:56.815237 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.815213 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:40:56.896994 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.896899 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-openshift-service-ca\") pod \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " Jul 08 09:40:56.896994 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.896957 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsvgg\" (UniqueName: \"kubernetes.io/projected/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-kube-api-access-gsvgg\") pod \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " Jul 08 09:40:56.896994 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.896992 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-maas-api-tls\") pod \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\" (UID: \"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c\") " Jul 08 09:40:56.897298 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.897264 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c" (UID: "02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:40:56.899111 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.899088 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-kube-api-access-gsvgg" (OuterVolumeSpecName: "kube-api-access-gsvgg") pod "02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c" (UID: "02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c"). InnerVolumeSpecName "kube-api-access-gsvgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:40:56.899206 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.899142 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c" (UID: "02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:40:56.997892 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.997859 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-openshift-service-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:40:56.997892 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.997887 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsvgg\" (UniqueName: \"kubernetes.io/projected/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-kube-api-access-gsvgg\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:40:56.997892 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:56.997899 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c-maas-api-tls\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:40:57.307086 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:57.307053 2572 generic.go:356] "Generic (PLEG): container finished" podID="02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c" containerID="0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65" exitCode=0 Jul 08 09:40:57.307273 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:57.307100 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" Jul 08 09:40:57.307273 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:57.307149 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" event={"ID":"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c","Type":"ContainerDied","Data":"0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65"} Jul 08 09:40:57.307273 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:57.307198 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw" event={"ID":"02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c","Type":"ContainerDied","Data":"6e85f673ab28d180118ea251212ff7ed767f2145c195ed5bc756f557e46aaf34"} Jul 08 09:40:57.307273 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:57.307220 2572 scope.go:117] "RemoveContainer" containerID="0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65" Jul 08 09:40:57.316109 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:57.316093 2572 scope.go:117] "RemoveContainer" containerID="0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65" Jul 08 09:40:57.316363 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:40:57.316348 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65\": container with ID starting with 0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65 not found: ID does not exist" containerID="0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65" Jul 08 09:40:57.316402 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:57.316371 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65"} err="failed to get container status \"0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65\": rpc error: code = NotFound desc = could not find container \"0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65\": container with ID starting with 0d4f03526c847587e95ce952e77167f9e6f571e06dd9670a0f48ec59d5309a65 not found: ID does not exist" Jul 08 09:40:57.329532 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:57.329504 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw"] Jul 08 09:40:57.334464 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:57.334441 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-802008-5dc49b97d6-zvfjw"] Jul 08 09:40:58.308045 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:40:58.308016 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c" path="/var/lib/kubelet/pods/02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c/volumes" Jul 08 09:41:00.186087 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.186065 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:41:00.322911 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.322883 2572 generic.go:356] "Generic (PLEG): container finished" podID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerID="6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d" exitCode=0 Jul 08 09:41:00.323079 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.322925 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" Jul 08 09:41:00.323079 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.322966 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" event={"ID":"b3712744-d17c-4b51-b5ac-f5c95d67e08b","Type":"ContainerDied","Data":"6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d"} Jul 08 09:41:00.323079 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.323013 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4" event={"ID":"b3712744-d17c-4b51-b5ac-f5c95d67e08b","Type":"ContainerDied","Data":"10176cf1b5de8adb6029b7ef4e6cd489cbd10969db78094a7f92e4e21d28ad90"} Jul 08 09:41:00.323079 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.323036 2572 scope.go:117] "RemoveContainer" containerID="6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d" Jul 08 09:41:00.328645 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.328616 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-credential-socket\") pod \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " Jul 08 09:41:00.328791 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.328670 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-token\") pod \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " Jul 08 09:41:00.328791 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.328698 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-podinfo\") pod \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " Jul 08 09:41:00.328791 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.328734 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-envoy\") pod \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " Jul 08 09:41:00.328791 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.328767 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-certs\") pod \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " Jul 08 09:41:00.329003 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.328835 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istiod-ca-cert\") pod \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " Jul 08 09:41:00.329003 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.328891 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zdz5\" (UniqueName: \"kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-kube-api-access-8zdz5\") pod \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " Jul 08 09:41:00.329003 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.328892 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b3712744-d17c-4b51-b5ac-f5c95d67e08b" (UID: "b3712744-d17c-4b51-b5ac-f5c95d67e08b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:00.329003 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.328921 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-data\") pod \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " Jul 08 09:41:00.329003 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.328964 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-socket\") pod \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\" (UID: \"b3712744-d17c-4b51-b5ac-f5c95d67e08b\") " Jul 08 09:41:00.329270 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.329011 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b3712744-d17c-4b51-b5ac-f5c95d67e08b" (UID: "b3712744-d17c-4b51-b5ac-f5c95d67e08b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:00.329270 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.329193 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b3712744-d17c-4b51-b5ac-f5c95d67e08b" (UID: "b3712744-d17c-4b51-b5ac-f5c95d67e08b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:41:00.329370 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.329292 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b3712744-d17c-4b51-b5ac-f5c95d67e08b" (UID: "b3712744-d17c-4b51-b5ac-f5c95d67e08b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:00.329370 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.329341 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-credential-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:00.329370 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.329358 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-certs\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:00.329965 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.329936 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-data" (OuterVolumeSpecName: "istio-data") pod "b3712744-d17c-4b51-b5ac-f5c95d67e08b" (UID: "b3712744-d17c-4b51-b5ac-f5c95d67e08b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:00.331083 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.331059 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b3712744-d17c-4b51-b5ac-f5c95d67e08b" (UID: "b3712744-d17c-4b51-b5ac-f5c95d67e08b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:41:00.331180 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.331065 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b3712744-d17c-4b51-b5ac-f5c95d67e08b" (UID: "b3712744-d17c-4b51-b5ac-f5c95d67e08b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:00.331180 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.331163 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-token" (OuterVolumeSpecName: "istio-token") pod "b3712744-d17c-4b51-b5ac-f5c95d67e08b" (UID: "b3712744-d17c-4b51-b5ac-f5c95d67e08b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:41:00.331258 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.331238 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-kube-api-access-8zdz5" (OuterVolumeSpecName: "kube-api-access-8zdz5") pod "b3712744-d17c-4b51-b5ac-f5c95d67e08b" (UID: "b3712744-d17c-4b51-b5ac-f5c95d67e08b"). InnerVolumeSpecName "kube-api-access-8zdz5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:41:00.332046 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.332028 2572 scope.go:117] "RemoveContainer" containerID="7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03" Jul 08 09:41:00.345949 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.345929 2572 scope.go:117] "RemoveContainer" containerID="6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d" Jul 08 09:41:00.346214 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:41:00.346196 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d\": container with ID starting with 6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d not found: ID does not exist" containerID="6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d" Jul 08 09:41:00.346280 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.346228 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d"} err="failed to get container status \"6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d\": rpc error: code = NotFound desc = could not find container \"6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d\": container with ID starting with 6db65ebacb64222918a70f75033b80532651db147eb9a993b639679ffcf0912d not found: ID does not exist" Jul 08 09:41:00.346280 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.346271 2572 scope.go:117] "RemoveContainer" containerID="7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03" Jul 08 09:41:00.346541 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:41:00.346525 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03\": container with ID starting with 7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03 not found: ID does not exist" containerID="7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03" Jul 08 09:41:00.346580 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.346547 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03"} err="failed to get container status \"7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03\": rpc error: code = NotFound desc = could not find container \"7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03\": container with ID starting with 7102d7e96ac074346be887bfffc329eec9f3bff51a694cccb7d52fcd8195ec03 not found: ID does not exist" Jul 08 09:41:00.430272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.430235 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zdz5\" (UniqueName: \"kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-kube-api-access-8zdz5\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:00.430272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.430265 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-data\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:00.430272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.430274 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-workload-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:00.430272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.430282 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-token\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:00.430533 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.430291 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-podinfo\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:00.430533 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.430299 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istio-envoy\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:00.430533 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.430306 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3712744-d17c-4b51-b5ac-f5c95d67e08b-istiod-ca-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:00.648899 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.648852 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4"] Jul 08 09:41:00.652353 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:00.652328 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2ec6b4-openshift-default-67f967f97c-2llq4"] Jul 08 09:41:02.308000 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:02.307970 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" path="/var/lib/kubelet/pods/b3712744-d17c-4b51-b5ac-f5c95d67e08b/volumes" Jul 08 09:41:03.252013 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.251986 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j"] Jul 08 09:41:03.252273 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252261 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41e3b748-9ddd-43dc-93ec-05144988f6c9" containerName="maas-api" Jul 08 09:41:03.252321 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252275 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="41e3b748-9ddd-43dc-93ec-05144988f6c9" containerName="maas-api" Jul 08 09:41:03.252321 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252285 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" Jul 08 09:41:03.252321 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252292 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" Jul 08 09:41:03.252321 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252299 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c" containerName="maas-api" Jul 08 09:41:03.252321 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252304 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c" containerName="maas-api" Jul 08 09:41:03.252468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252326 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" Jul 08 09:41:03.252468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252331 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" Jul 08 09:41:03.252468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252389 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" Jul 08 09:41:03.252468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252395 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="02c0ad5f-ca6f-40bc-a16d-d5e3abd06a7c" containerName="maas-api" Jul 08 09:41:03.252468 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.252401 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="41e3b748-9ddd-43dc-93ec-05144988f6c9" containerName="maas-api" Jul 08 09:41:03.256550 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.256525 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.260663 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.260643 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-7250c4-openshift-default-dockercfg-hmnsq\"" Jul 08 09:41:03.268229 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.268208 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j"] Jul 08 09:41:03.356856 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.356827 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-token\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.357290 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.356867 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-envoy\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.357290 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.356913 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-data\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.357290 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.356935 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-socket\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.357290 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.356953 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-credential-socket\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.357290 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.356967 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-podinfo\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.357290 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.356993 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-certs\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.357290 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.357024 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1edbad71-4f6e-473b-993e-bcecda3b03d2-istiod-ca-cert\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.357290 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.357043 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s4p5\" (UniqueName: \"kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-kube-api-access-2s4p5\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458087 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458050 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-data\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458087 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458088 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-socket\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458353 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458106 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-credential-socket\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458353 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458224 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-podinfo\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458353 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458266 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-certs\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458353 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458302 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1edbad71-4f6e-473b-993e-bcecda3b03d2-istiod-ca-cert\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458353 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458337 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s4p5\" (UniqueName: \"kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-kube-api-access-2s4p5\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458595 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458418 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-token\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458595 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458458 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-envoy\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458595 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458457 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-socket\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458595 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458569 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-credential-socket\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458816 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458695 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-certs\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.458901 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.458882 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-data\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.459025 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.459008 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1edbad71-4f6e-473b-993e-bcecda3b03d2-istiod-ca-cert\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.460603 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.460582 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-envoy\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.460852 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.460834 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-podinfo\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.469173 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.469148 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-token\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.469736 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.469712 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s4p5\" (UniqueName: \"kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-kube-api-access-2s4p5\") pod \"e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.567307 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.567213 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:03.703243 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.703214 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j"] Jul 08 09:41:03.705605 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:41:03.705579 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edbad71_4f6e_473b_993e_bcecda3b03d2.slice/crio-2e2a004b59112a37572019b94371b37dfe389453b1826bf2d6daddd3a4e782b0 WatchSource:0}: Error finding container 2e2a004b59112a37572019b94371b37dfe389453b1826bf2d6daddd3a4e782b0: Status 404 returned error can't find the container with id 2e2a004b59112a37572019b94371b37dfe389453b1826bf2d6daddd3a4e782b0 Jul 08 09:41:03.707549 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.707517 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:03.707622 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.707584 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:03.707622 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:03.707612 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:04.342846 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:04.342763 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" event={"ID":"1edbad71-4f6e-473b-993e-bcecda3b03d2","Type":"ContainerStarted","Data":"56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81"} Jul 08 09:41:04.342846 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:04.342823 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" event={"ID":"1edbad71-4f6e-473b-993e-bcecda3b03d2","Type":"ContainerStarted","Data":"2e2a004b59112a37572019b94371b37dfe389453b1826bf2d6daddd3a4e782b0"} Jul 08 09:41:04.368427 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:04.368379 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" podStartSLOduration=1.36836188 podStartE2EDuration="1.36836188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:41:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:41:04.36808209 +0000 UTC m=+2062.618041372" watchObservedRunningTime="2026-07-08 09:41:04.36836188 +0000 UTC m=+2062.618321161" Jul 08 09:41:04.568000 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:04.567961 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:04.572913 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:04.572889 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:05.353096 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:05.353063 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:05.353974 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:05.353953 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:06.454100 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.454067 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj"] Jul 08 09:41:06.454477 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.454395 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3712744-d17c-4b51-b5ac-f5c95d67e08b" containerName="istio-proxy" Jul 08 09:41:06.457396 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.457358 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:06.460791 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.460770 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-7250c4-serving-cert\"" Jul 08 09:41:06.472627 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.472597 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj"] Jul 08 09:41:06.583712 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.583675 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9b90ca9-5164-440e-a566-391bfdd400b0-openshift-service-ca\") pod \"maas-api-e2e-api-a-7250c4-7964db5f47-pszmj\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:06.583905 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.583737 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b90ca9-5164-440e-a566-391bfdd400b0-maas-api-tls\") pod \"maas-api-e2e-api-a-7250c4-7964db5f47-pszmj\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:06.583905 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.583843 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djgrv\" (UniqueName: \"kubernetes.io/projected/d9b90ca9-5164-440e-a566-391bfdd400b0-kube-api-access-djgrv\") pod \"maas-api-e2e-api-a-7250c4-7964db5f47-pszmj\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:06.685222 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.685180 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9b90ca9-5164-440e-a566-391bfdd400b0-openshift-service-ca\") pod \"maas-api-e2e-api-a-7250c4-7964db5f47-pszmj\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:06.685416 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.685247 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b90ca9-5164-440e-a566-391bfdd400b0-maas-api-tls\") pod \"maas-api-e2e-api-a-7250c4-7964db5f47-pszmj\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:06.685416 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.685276 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djgrv\" (UniqueName: \"kubernetes.io/projected/d9b90ca9-5164-440e-a566-391bfdd400b0-kube-api-access-djgrv\") pod \"maas-api-e2e-api-a-7250c4-7964db5f47-pszmj\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:06.685490 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:41:06.685410 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-7250c4-serving-cert: secret "maas-api-e2e-api-a-7250c4-serving-cert" not found Jul 08 09:41:06.685490 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:41:06.685473 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9b90ca9-5164-440e-a566-391bfdd400b0-maas-api-tls podName:d9b90ca9-5164-440e-a566-391bfdd400b0 nodeName:}" failed. No retries permitted until 2026-07-08 09:41:07.185452757 +0000 UTC m=+2065.435412024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d9b90ca9-5164-440e-a566-391bfdd400b0-maas-api-tls") pod "maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" (UID: "d9b90ca9-5164-440e-a566-391bfdd400b0") : secret "maas-api-e2e-api-a-7250c4-serving-cert" not found Jul 08 09:41:06.685891 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.685870 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9b90ca9-5164-440e-a566-391bfdd400b0-openshift-service-ca\") pod \"maas-api-e2e-api-a-7250c4-7964db5f47-pszmj\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:06.695149 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:06.695120 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djgrv\" (UniqueName: \"kubernetes.io/projected/d9b90ca9-5164-440e-a566-391bfdd400b0-kube-api-access-djgrv\") pod \"maas-api-e2e-api-a-7250c4-7964db5f47-pszmj\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:07.191013 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:07.190980 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b90ca9-5164-440e-a566-391bfdd400b0-maas-api-tls\") pod \"maas-api-e2e-api-a-7250c4-7964db5f47-pszmj\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:07.193394 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:07.193375 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b90ca9-5164-440e-a566-391bfdd400b0-maas-api-tls\") pod \"maas-api-e2e-api-a-7250c4-7964db5f47-pszmj\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:07.370682 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:07.370656 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:07.511847 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:07.511827 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj"] Jul 08 09:41:07.513996 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:41:07.513965 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b90ca9_5164_440e_a566_391bfdd400b0.slice/crio-6bacd7113f6749106a1f399a2063433b0751c338e840d454c7d0bbf6ffd4af0a WatchSource:0}: Error finding container 6bacd7113f6749106a1f399a2063433b0751c338e840d454c7d0bbf6ffd4af0a: Status 404 returned error can't find the container with id 6bacd7113f6749106a1f399a2063433b0751c338e840d454c7d0bbf6ffd4af0a Jul 08 09:41:08.368293 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:08.368270 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" event={"ID":"d9b90ca9-5164-440e-a566-391bfdd400b0","Type":"ContainerStarted","Data":"6bacd7113f6749106a1f399a2063433b0751c338e840d454c7d0bbf6ffd4af0a"} Jul 08 09:41:09.374413 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:09.374378 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" event={"ID":"d9b90ca9-5164-440e-a566-391bfdd400b0","Type":"ContainerStarted","Data":"de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f"} Jul 08 09:41:09.374864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:09.374464 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:09.396898 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:09.396851 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" podStartSLOduration=2.550047661 podStartE2EDuration="3.396838299s" totalImagesPullingTime="846.790638ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:41:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:41:09.395712916 +0000 UTC m=+2067.645672200" watchObservedRunningTime="2026-07-08 09:41:09.396838299 +0000 UTC m=+2067.646797580" Jul 08 09:41:11.724729 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.724688 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9"] Jul 08 09:41:11.728162 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.728142 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.730992 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.730970 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-4368cb-openshift-default-dockercfg-49bfj\"" Jul 08 09:41:11.744087 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.744062 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9"] Jul 08 09:41:11.831738 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.831703 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-token\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.831944 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.831774 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-certs\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.831944 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.831800 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-envoy\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.831944 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.831841 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-credential-socket\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.831944 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.831863 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bstq\" (UniqueName: \"kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-kube-api-access-9bstq\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.831944 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.831896 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-data\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.831944 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.831922 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-podinfo\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.832177 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.831951 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istiod-ca-cert\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.832177 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.831984 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-socket\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933105 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933057 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-credential-socket\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933105 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933104 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bstq\" (UniqueName: \"kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-kube-api-access-9bstq\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933135 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-data\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933155 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-podinfo\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933189 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istiod-ca-cert\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933212 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-socket\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933271 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-token\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933315 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-certs\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933369 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933336 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-envoy\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933692 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933569 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-data\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933692 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933592 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-credential-socket\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933800 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933702 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-socket\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933800 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933776 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-certs\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.933994 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.933976 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istiod-ca-cert\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.935558 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.935529 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-envoy\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.935679 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.935668 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-podinfo\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.943677 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.943656 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-token\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:11.943881 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:11.943680 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bstq\" (UniqueName: \"kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-kube-api-access-9bstq\") pod \"e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:12.039018 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:12.038979 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:12.177479 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:12.177437 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9"] Jul 08 09:41:12.178327 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:41:12.178297 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961b8e8_b909_43cf_bb43_5a36c0eb563b.slice/crio-a5ff6bc43ebecbc2835833a84eab2d1b99e791094129711ea8be1e0d386ed644 WatchSource:0}: Error finding container a5ff6bc43ebecbc2835833a84eab2d1b99e791094129711ea8be1e0d386ed644: Status 404 returned error can't find the container with id a5ff6bc43ebecbc2835833a84eab2d1b99e791094129711ea8be1e0d386ed644 Jul 08 09:41:12.180495 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:12.180454 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:12.180588 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:12.180539 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:12.180588 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:12.180578 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:12.390379 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:12.390273 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" event={"ID":"3961b8e8-b909-43cf-bb43-5a36c0eb563b","Type":"ContainerStarted","Data":"395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce"} Jul 08 09:41:12.390379 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:12.390330 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" event={"ID":"3961b8e8-b909-43cf-bb43-5a36c0eb563b","Type":"ContainerStarted","Data":"a5ff6bc43ebecbc2835833a84eab2d1b99e791094129711ea8be1e0d386ed644"} Jul 08 09:41:12.420375 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:12.420309 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" podStartSLOduration=1.420288738 podStartE2EDuration="1.420288738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:41:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:41:12.416845755 +0000 UTC m=+2070.666805038" watchObservedRunningTime="2026-07-08 09:41:12.420288738 +0000 UTC m=+2070.670248021" Jul 08 09:41:13.039635 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:13.039604 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:13.044700 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:13.044668 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:13.399901 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:13.399798 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:13.401509 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:13.401475 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:15.395090 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:15.395057 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:20.355034 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:20.354992 2572 patch_prober.go:28] interesting pod/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 08 09:41:20.355517 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:20.355056 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 08 09:41:20.440733 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:20.440701 2572 generic.go:356] "Generic (PLEG): container finished" podID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerID="56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81" exitCode=0 Jul 08 09:41:20.440917 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:20.440771 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" event={"ID":"1edbad71-4f6e-473b-993e-bcecda3b03d2","Type":"ContainerDied","Data":"56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81"} Jul 08 09:41:20.441163 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:20.441149 2572 scope.go:117] "RemoveContainer" containerID="56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81" Jul 08 09:41:20.442201 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:20.442173 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:20.442271 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:20.442254 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:20.442320 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:20.442305 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:21.447693 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:21.447660 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" event={"ID":"1edbad71-4f6e-473b-993e-bcecda3b03d2","Type":"ContainerStarted","Data":"dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7"} Jul 08 09:41:21.567582 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:21.567540 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:21.567786 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:21.567741 2572 patch_prober.go:28] interesting pod/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 08 09:41:21.567846 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:21.567814 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 08 09:41:22.567878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:22.567839 2572 patch_prober.go:28] interesting pod/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 08 09:41:22.568306 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:22.567910 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 08 09:41:23.568228 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:23.568187 2572 patch_prober.go:28] interesting pod/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 08 09:41:23.568644 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:23.568248 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 08 09:41:24.568335 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:24.568293 2572 patch_prober.go:28] interesting pod/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 08 09:41:24.568733 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:24.568352 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 08 09:41:25.572178 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:25.572149 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:25.572668 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:25.572524 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:25.573131 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:25.573113 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:27.973587 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:27.973553 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj"] Jul 08 09:41:27.974128 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:27.973834 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" podUID="d9b90ca9-5164-440e-a566-391bfdd400b0" containerName="maas-api" containerID="cri-o://de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f" gracePeriod=30 Jul 08 09:41:28.236076 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.236052 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:28.281120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.281086 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djgrv\" (UniqueName: \"kubernetes.io/projected/d9b90ca9-5164-440e-a566-391bfdd400b0-kube-api-access-djgrv\") pod \"d9b90ca9-5164-440e-a566-391bfdd400b0\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " Jul 08 09:41:28.281344 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.281160 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b90ca9-5164-440e-a566-391bfdd400b0-maas-api-tls\") pod \"d9b90ca9-5164-440e-a566-391bfdd400b0\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " Jul 08 09:41:28.281344 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.281218 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9b90ca9-5164-440e-a566-391bfdd400b0-openshift-service-ca\") pod \"d9b90ca9-5164-440e-a566-391bfdd400b0\" (UID: \"d9b90ca9-5164-440e-a566-391bfdd400b0\") " Jul 08 09:41:28.281910 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.281871 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9b90ca9-5164-440e-a566-391bfdd400b0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9b90ca9-5164-440e-a566-391bfdd400b0" (UID: "d9b90ca9-5164-440e-a566-391bfdd400b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:41:28.283328 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.283301 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9b90ca9-5164-440e-a566-391bfdd400b0-kube-api-access-djgrv" (OuterVolumeSpecName: "kube-api-access-djgrv") pod "d9b90ca9-5164-440e-a566-391bfdd400b0" (UID: "d9b90ca9-5164-440e-a566-391bfdd400b0"). InnerVolumeSpecName "kube-api-access-djgrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:41:28.283545 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.283517 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9b90ca9-5164-440e-a566-391bfdd400b0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9b90ca9-5164-440e-a566-391bfdd400b0" (UID: "d9b90ca9-5164-440e-a566-391bfdd400b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:41:28.382249 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.382214 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djgrv\" (UniqueName: \"kubernetes.io/projected/d9b90ca9-5164-440e-a566-391bfdd400b0-kube-api-access-djgrv\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:28.382249 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.382246 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b90ca9-5164-440e-a566-391bfdd400b0-maas-api-tls\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:28.382249 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.382255 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9b90ca9-5164-440e-a566-391bfdd400b0-openshift-service-ca\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:28.483813 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.483777 2572 generic.go:356] "Generic (PLEG): container finished" podID="d9b90ca9-5164-440e-a566-391bfdd400b0" containerID="de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f" exitCode=0 Jul 08 09:41:28.483973 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.483873 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" Jul 08 09:41:28.483973 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.483888 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" event={"ID":"d9b90ca9-5164-440e-a566-391bfdd400b0","Type":"ContainerDied","Data":"de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f"} Jul 08 09:41:28.483973 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.483932 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj" event={"ID":"d9b90ca9-5164-440e-a566-391bfdd400b0","Type":"ContainerDied","Data":"6bacd7113f6749106a1f399a2063433b0751c338e840d454c7d0bbf6ffd4af0a"} Jul 08 09:41:28.483973 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.483954 2572 scope.go:117] "RemoveContainer" containerID="de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f" Jul 08 09:41:28.493715 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.493697 2572 scope.go:117] "RemoveContainer" containerID="de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f" Jul 08 09:41:28.494055 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:41:28.494034 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f\": container with ID starting with de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f not found: ID does not exist" containerID="de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f" Jul 08 09:41:28.494133 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.494063 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f"} err="failed to get container status \"de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f\": rpc error: code = NotFound desc = could not find container \"de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f\": container with ID starting with de6fb9d7ddb8a91e2c71a9eacb1ce1338e019fa3dc18616b6fc52386b6a4589f not found: ID does not exist" Jul 08 09:41:28.503194 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.503161 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj"] Jul 08 09:41:28.511184 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:28.511160 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-7250c4-7964db5f47-pszmj"] Jul 08 09:41:29.441943 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:29.441900 2572 patch_prober.go:28] interesting pod/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 09:41:29.442384 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:29.441980 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 09:41:30.307493 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:30.307461 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9b90ca9-5164-440e-a566-391bfdd400b0" path="/var/lib/kubelet/pods/d9b90ca9-5164-440e-a566-391bfdd400b0/volumes" Jul 08 09:41:30.496514 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:30.496481 2572 generic.go:356] "Generic (PLEG): container finished" podID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerID="395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce" exitCode=0 Jul 08 09:41:30.496876 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:30.496551 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" event={"ID":"3961b8e8-b909-43cf-bb43-5a36c0eb563b","Type":"ContainerDied","Data":"395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce"} Jul 08 09:41:30.496964 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:30.496950 2572 scope.go:117] "RemoveContainer" containerID="395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce" Jul 08 09:41:30.497909 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:30.497886 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:30.497965 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:30.497949 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:30.497999 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:30.497980 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:31.506264 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:31.506220 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" event={"ID":"3961b8e8-b909-43cf-bb43-5a36c0eb563b","Type":"ContainerStarted","Data":"e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1"} Jul 08 09:41:32.039405 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:32.039367 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:32.039619 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:32.039600 2572 patch_prober.go:28] interesting pod/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 08 09:41:32.039666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:32.039643 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 08 09:41:33.040375 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:33.040336 2572 patch_prober.go:28] interesting pod/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 08 09:41:33.040862 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:33.040418 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 08 09:41:34.040041 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:34.040007 2572 patch_prober.go:28] interesting pod/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 08 09:41:34.040228 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:34.040067 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 08 09:41:34.437352 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:34.437247 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j"] Jul 08 09:41:34.437955 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:34.437927 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" containerID="cri-o://dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7" gracePeriod=30 Jul 08 09:41:35.040301 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:35.040258 2572 patch_prober.go:28] interesting pod/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 08 09:41:35.040498 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:35.040330 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 08 09:41:36.043403 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:36.043362 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:36.043878 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:36.043662 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:36.044904 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:36.044877 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:39.691447 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.691423 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:39.780518 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780488 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-podinfo\") pod \"1edbad71-4f6e-473b-993e-bcecda3b03d2\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " Jul 08 09:41:39.780724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780551 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1edbad71-4f6e-473b-993e-bcecda3b03d2-istiod-ca-cert\") pod \"1edbad71-4f6e-473b-993e-bcecda3b03d2\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " Jul 08 09:41:39.780724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780594 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-credential-socket\") pod \"1edbad71-4f6e-473b-993e-bcecda3b03d2\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " Jul 08 09:41:39.780724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780620 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-token\") pod \"1edbad71-4f6e-473b-993e-bcecda3b03d2\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " Jul 08 09:41:39.780724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780648 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s4p5\" (UniqueName: \"kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-kube-api-access-2s4p5\") pod \"1edbad71-4f6e-473b-993e-bcecda3b03d2\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " Jul 08 09:41:39.780724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780674 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-data\") pod \"1edbad71-4f6e-473b-993e-bcecda3b03d2\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " Jul 08 09:41:39.780724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780708 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-envoy\") pod \"1edbad71-4f6e-473b-993e-bcecda3b03d2\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " Jul 08 09:41:39.781049 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780741 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-certs\") pod \"1edbad71-4f6e-473b-993e-bcecda3b03d2\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " Jul 08 09:41:39.781049 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780823 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-socket\") pod \"1edbad71-4f6e-473b-993e-bcecda3b03d2\" (UID: \"1edbad71-4f6e-473b-993e-bcecda3b03d2\") " Jul 08 09:41:39.781049 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780931 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1edbad71-4f6e-473b-993e-bcecda3b03d2" (UID: "1edbad71-4f6e-473b-993e-bcecda3b03d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:39.781049 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.780947 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1edbad71-4f6e-473b-993e-bcecda3b03d2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1edbad71-4f6e-473b-993e-bcecda3b03d2" (UID: "1edbad71-4f6e-473b-993e-bcecda3b03d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:41:39.781248 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.781070 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1edbad71-4f6e-473b-993e-bcecda3b03d2" (UID: "1edbad71-4f6e-473b-993e-bcecda3b03d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:39.781248 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.781167 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1edbad71-4f6e-473b-993e-bcecda3b03d2" (UID: "1edbad71-4f6e-473b-993e-bcecda3b03d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:39.781248 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.781137 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1edbad71-4f6e-473b-993e-bcecda3b03d2-istiod-ca-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:39.781248 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.781199 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-credential-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:39.781644 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.781621 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-data" (OuterVolumeSpecName: "istio-data") pod "1edbad71-4f6e-473b-993e-bcecda3b03d2" (UID: "1edbad71-4f6e-473b-993e-bcecda3b03d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:39.782981 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.782961 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-kube-api-access-2s4p5" (OuterVolumeSpecName: "kube-api-access-2s4p5") pod "1edbad71-4f6e-473b-993e-bcecda3b03d2" (UID: "1edbad71-4f6e-473b-993e-bcecda3b03d2"). InnerVolumeSpecName "kube-api-access-2s4p5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:41:39.783096 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.783006 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1edbad71-4f6e-473b-993e-bcecda3b03d2" (UID: "1edbad71-4f6e-473b-993e-bcecda3b03d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:39.783210 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.783193 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-token" (OuterVolumeSpecName: "istio-token") pod "1edbad71-4f6e-473b-993e-bcecda3b03d2" (UID: "1edbad71-4f6e-473b-993e-bcecda3b03d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:41:39.783268 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.783211 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1edbad71-4f6e-473b-993e-bcecda3b03d2" (UID: "1edbad71-4f6e-473b-993e-bcecda3b03d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:41:39.881954 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.881913 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-token\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:39.881954 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.881946 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s4p5\" (UniqueName: \"kubernetes.io/projected/1edbad71-4f6e-473b-993e-bcecda3b03d2-kube-api-access-2s4p5\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:39.881954 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.881959 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-data\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:39.882199 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.881969 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-envoy\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:39.882199 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.881993 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-certs\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:39.882199 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.882002 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edbad71-4f6e-473b-993e-bcecda3b03d2-workload-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:39.882199 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:39.882009 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1edbad71-4f6e-473b-993e-bcecda3b03d2-istio-podinfo\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:40.552556 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.552518 2572 generic.go:356] "Generic (PLEG): container finished" podID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerID="dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7" exitCode=0 Jul 08 09:41:40.552724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.552573 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" event={"ID":"1edbad71-4f6e-473b-993e-bcecda3b03d2","Type":"ContainerDied","Data":"dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7"} Jul 08 09:41:40.552724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.552619 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" event={"ID":"1edbad71-4f6e-473b-993e-bcecda3b03d2","Type":"ContainerDied","Data":"2e2a004b59112a37572019b94371b37dfe389453b1826bf2d6daddd3a4e782b0"} Jul 08 09:41:40.552724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.552641 2572 scope.go:117] "RemoveContainer" containerID="dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7" Jul 08 09:41:40.552724 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.552581 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j" Jul 08 09:41:40.561475 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.561217 2572 scope.go:117] "RemoveContainer" containerID="56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81" Jul 08 09:41:40.568911 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.568890 2572 scope.go:117] "RemoveContainer" containerID="dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7" Jul 08 09:41:40.569141 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:41:40.569119 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7\": container with ID starting with dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7 not found: ID does not exist" containerID="dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7" Jul 08 09:41:40.569203 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.569149 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7"} err="failed to get container status \"dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7\": rpc error: code = NotFound desc = could not find container \"dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7\": container with ID starting with dc054009f9139f4c16e0c0c960e0cbac34121c3e22804e68055ccadcd3d590a7 not found: ID does not exist" Jul 08 09:41:40.569203 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.569165 2572 scope.go:117] "RemoveContainer" containerID="56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81" Jul 08 09:41:40.569428 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:41:40.569408 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81\": container with ID starting with 56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81 not found: ID does not exist" containerID="56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81" Jul 08 09:41:40.569469 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.569435 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81"} err="failed to get container status \"56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81\": rpc error: code = NotFound desc = could not find container \"56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81\": container with ID starting with 56950fd254099de0a0f8237e2ca64fb5a6afb06e56b3b13e7cea34244e6eac81 not found: ID does not exist" Jul 08 09:41:40.576895 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.576874 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j"] Jul 08 09:41:40.584524 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:40.584504 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-7250c4-openshift-default-56bdcccb9-kfr4j"] Jul 08 09:41:41.727720 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:41.727685 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9"] Jul 08 09:41:41.728164 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:41.727910 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" containerID="cri-o://e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1" gracePeriod=30 Jul 08 09:41:42.307406 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.307374 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" path="/var/lib/kubelet/pods/1edbad71-4f6e-473b-993e-bcecda3b03d2/volumes" Jul 08 09:41:42.630114 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.630038 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f"] Jul 08 09:41:42.630384 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.630368 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9b90ca9-5164-440e-a566-391bfdd400b0" containerName="maas-api" Jul 08 09:41:42.630434 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.630385 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9b90ca9-5164-440e-a566-391bfdd400b0" containerName="maas-api" Jul 08 09:41:42.630434 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.630399 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" Jul 08 09:41:42.630434 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.630404 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" Jul 08 09:41:42.630434 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.630413 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" Jul 08 09:41:42.630434 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.630419 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" Jul 08 09:41:42.630595 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.630472 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9b90ca9-5164-440e-a566-391bfdd400b0" containerName="maas-api" Jul 08 09:41:42.630595 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.630493 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" Jul 08 09:41:42.630595 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.630499 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edbad71-4f6e-473b-993e-bcecda3b03d2" containerName="istio-proxy" Jul 08 09:41:42.632590 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.632574 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.636792 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.636742 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8c4427-openshift-default-dockercfg-swspb\"" Jul 08 09:41:42.655618 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.655591 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f"] Jul 08 09:41:42.704386 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.704349 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/232dc30c-7f20-403d-add2-fe8c6c0042f1-istiod-ca-cert\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.704386 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.704390 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-certs\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.704589 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.704414 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-podinfo\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.704589 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.704479 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-envoy\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.704589 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.704527 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-data\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.704589 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.704542 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-token\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.704589 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.704572 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw5g9\" (UniqueName: \"kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-kube-api-access-vw5g9\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.704797 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.704607 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-socket\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.704797 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.704642 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-credential-socket\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805321 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw5g9\" (UniqueName: \"kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-kube-api-access-vw5g9\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805364 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805371 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-socket\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805404 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-credential-socket\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805437 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/232dc30c-7f20-403d-add2-fe8c6c0042f1-istiod-ca-cert\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805458 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-certs\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805490 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-podinfo\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805524 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-envoy\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805556 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-data\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805575 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-token\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805857 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-socket\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.805920 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805918 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-credential-socket\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.806329 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.805988 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-certs\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.806329 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.806001 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-data\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.806396 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.806337 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/232dc30c-7f20-403d-add2-fe8c6c0042f1-istiod-ca-cert\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.807835 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.807809 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-envoy\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.807924 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.807909 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-podinfo\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.815734 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.815711 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw5g9\" (UniqueName: \"kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-kube-api-access-vw5g9\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.818474 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.818452 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-token\") pod \"e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:42.942639 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:42.942559 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:43.085446 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:43.085421 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f"] Jul 08 09:41:43.087442 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:41:43.087415 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232dc30c_7f20_403d_add2_fe8c6c0042f1.slice/crio-38334e94087a8d1b44528153b0791eccfc5ea62a1adc7390f12a67449dda75ee WatchSource:0}: Error finding container 38334e94087a8d1b44528153b0791eccfc5ea62a1adc7390f12a67449dda75ee: Status 404 returned error can't find the container with id 38334e94087a8d1b44528153b0791eccfc5ea62a1adc7390f12a67449dda75ee Jul 08 09:41:43.089480 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:43.089445 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:43.089564 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:43.089514 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:43.089564 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:43.089545 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:43.569200 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:43.569167 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" event={"ID":"232dc30c-7f20-403d-add2-fe8c6c0042f1","Type":"ContainerStarted","Data":"9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255"} Jul 08 09:41:43.569434 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:43.569223 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" event={"ID":"232dc30c-7f20-403d-add2-fe8c6c0042f1","Type":"ContainerStarted","Data":"38334e94087a8d1b44528153b0791eccfc5ea62a1adc7390f12a67449dda75ee"} Jul 08 09:41:43.600494 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:43.600442 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" podStartSLOduration=1.6004244989999998 podStartE2EDuration="1.600424499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:41:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:41:43.599992053 +0000 UTC m=+2101.849951348" watchObservedRunningTime="2026-07-08 09:41:43.600424499 +0000 UTC m=+2101.850383781" Jul 08 09:41:43.943451 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:43.943366 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:44.948045 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:44.948018 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:45.578714 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:45.578682 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:45.579782 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:45.579742 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:41:46.971704 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:46.971678 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:47.041547 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.041458 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bstq\" (UniqueName: \"kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-kube-api-access-9bstq\") pod \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " Jul 08 09:41:47.041547 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.041539 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-envoy\") pod \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " Jul 08 09:41:47.041822 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.041561 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-podinfo\") pod \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " Jul 08 09:41:47.041822 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.041587 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-data\") pod \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " Jul 08 09:41:47.041822 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.041622 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-token\") pod \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " Jul 08 09:41:47.041822 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.041672 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-socket\") pod \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " Jul 08 09:41:47.041822 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.041703 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-certs\") pod \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " Jul 08 09:41:47.041822 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.041735 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istiod-ca-cert\") pod \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " Jul 08 09:41:47.041822 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.041790 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-credential-socket\") pod \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\" (UID: \"3961b8e8-b909-43cf-bb43-5a36c0eb563b\") " Jul 08 09:41:47.042251 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.042160 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3961b8e8-b909-43cf-bb43-5a36c0eb563b" (UID: "3961b8e8-b909-43cf-bb43-5a36c0eb563b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:47.042337 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.042302 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3961b8e8-b909-43cf-bb43-5a36c0eb563b" (UID: "3961b8e8-b909-43cf-bb43-5a36c0eb563b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:47.042536 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.042511 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3961b8e8-b909-43cf-bb43-5a36c0eb563b" (UID: "3961b8e8-b909-43cf-bb43-5a36c0eb563b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:41:47.043049 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.043024 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3961b8e8-b909-43cf-bb43-5a36c0eb563b" (UID: "3961b8e8-b909-43cf-bb43-5a36c0eb563b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:47.043318 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.043296 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3961b8e8-b909-43cf-bb43-5a36c0eb563b" (UID: "3961b8e8-b909-43cf-bb43-5a36c0eb563b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:47.043906 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.043882 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-kube-api-access-9bstq" (OuterVolumeSpecName: "kube-api-access-9bstq") pod "3961b8e8-b909-43cf-bb43-5a36c0eb563b" (UID: "3961b8e8-b909-43cf-bb43-5a36c0eb563b"). InnerVolumeSpecName "kube-api-access-9bstq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:41:47.043990 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.043910 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3961b8e8-b909-43cf-bb43-5a36c0eb563b" (UID: "3961b8e8-b909-43cf-bb43-5a36c0eb563b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:41:47.044152 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.044138 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3961b8e8-b909-43cf-bb43-5a36c0eb563b" (UID: "3961b8e8-b909-43cf-bb43-5a36c0eb563b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:47.044203 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.044138 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3961b8e8-b909-43cf-bb43-5a36c0eb563b" (UID: "3961b8e8-b909-43cf-bb43-5a36c0eb563b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:41:47.142980 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.142940 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-podinfo\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:47.142980 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.142973 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-data\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:47.142980 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.142981 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-token\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:47.142980 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.142989 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:47.143246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.142996 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-workload-certs\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:47.143246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.143004 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istiod-ca-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:47.143246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.143012 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-credential-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:47.143246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.143021 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bstq\" (UniqueName: \"kubernetes.io/projected/3961b8e8-b909-43cf-bb43-5a36c0eb563b-kube-api-access-9bstq\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:47.143246 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.143029 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3961b8e8-b909-43cf-bb43-5a36c0eb563b-istio-envoy\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:41:47.589530 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.589500 2572 generic.go:356] "Generic (PLEG): container finished" podID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerID="e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1" exitCode=0 Jul 08 09:41:47.589721 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.589547 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" Jul 08 09:41:47.589721 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.589592 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" event={"ID":"3961b8e8-b909-43cf-bb43-5a36c0eb563b","Type":"ContainerDied","Data":"e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1"} Jul 08 09:41:47.589721 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.589645 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9" event={"ID":"3961b8e8-b909-43cf-bb43-5a36c0eb563b","Type":"ContainerDied","Data":"a5ff6bc43ebecbc2835833a84eab2d1b99e791094129711ea8be1e0d386ed644"} Jul 08 09:41:47.589721 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.589668 2572 scope.go:117] "RemoveContainer" containerID="e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1" Jul 08 09:41:47.598969 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.598913 2572 scope.go:117] "RemoveContainer" containerID="395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce" Jul 08 09:41:47.606688 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.606671 2572 scope.go:117] "RemoveContainer" containerID="e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1" Jul 08 09:41:47.606995 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:41:47.606976 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1\": container with ID starting with e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1 not found: ID does not exist" containerID="e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1" Jul 08 09:41:47.607083 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.607008 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1"} err="failed to get container status \"e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1\": rpc error: code = NotFound desc = could not find container \"e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1\": container with ID starting with e3f239ff1ae178b24f2e7a16915848e8a18793c0b12c7e13b9b3f720525bbbb1 not found: ID does not exist" Jul 08 09:41:47.607083 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.607031 2572 scope.go:117] "RemoveContainer" containerID="395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce" Jul 08 09:41:47.607297 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:41:47.607281 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce\": container with ID starting with 395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce not found: ID does not exist" containerID="395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce" Jul 08 09:41:47.607336 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.607304 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce"} err="failed to get container status \"395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce\": rpc error: code = NotFound desc = could not find container \"395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce\": container with ID starting with 395752090d6c94e572efa5d87e958bb233a66a7be7655f2d6b200cd9861d52ce not found: ID does not exist" Jul 08 09:41:47.620725 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.620700 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9"] Jul 08 09:41:47.622944 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:47.622923 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4368cb-openshift-default-65d7dcdd57-wkxd9"] Jul 08 09:41:48.307368 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:48.307328 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" path="/var/lib/kubelet/pods/3961b8e8-b909-43cf-bb43-5a36c0eb563b/volumes" Jul 08 09:41:50.955940 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:50.955903 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5"] Jul 08 09:41:50.956418 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:50.956324 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" Jul 08 09:41:50.956418 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:50.956341 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" Jul 08 09:41:50.956418 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:50.956380 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" Jul 08 09:41:50.956418 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:50.956387 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" Jul 08 09:41:50.956621 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:50.956463 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" Jul 08 09:41:50.960993 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:50.960970 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:50.963696 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:50.963676 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f0ff0b-openshift-default-dockercfg-ttbnq\"" Jul 08 09:41:50.977205 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:50.977180 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5"] Jul 08 09:41:51.080504 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.080468 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-podinfo\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.080504 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.080507 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hznvj\" (UniqueName: \"kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-kube-api-access-hznvj\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.080779 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.080531 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-data\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.080779 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.080560 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istiod-ca-cert\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.080779 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.080660 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-credential-socket\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.080779 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.080701 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-socket\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.080779 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.080776 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-certs\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.080977 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.080829 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-envoy\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.080977 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.080871 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-token\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182145 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182099 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-data\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182359 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istiod-ca-cert\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182359 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182211 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-credential-socket\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182359 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182240 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-socket\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182543 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182376 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-certs\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182543 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-envoy\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182543 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182492 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-token\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182688 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182568 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-data\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182688 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182596 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-socket\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182688 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182630 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-podinfo\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182688 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182654 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-credential-socket\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.182688 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182673 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hznvj\" (UniqueName: \"kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-kube-api-access-hznvj\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.183043 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.182893 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-certs\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.183124 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.183098 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istiod-ca-cert\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.184741 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.184716 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-envoy\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.184883 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.184868 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-podinfo\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.195863 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.195833 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-token\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.195991 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.195954 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hznvj\" (UniqueName: \"kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-kube-api-access-hznvj\") pod \"e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.270622 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.270586 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:51.423882 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.423857 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5"] Jul 08 09:41:51.426319 ip-10-0-136-68 kubenswrapper[2572]: W0708 09:41:51.426291 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76ee0d61_5c67_4de4_afd6_25724ca0a0a0.slice/crio-488e01f9a329463fe612f71457c1a5cc270a73e19893f710e0c5aea4606f30ec WatchSource:0}: Error finding container 488e01f9a329463fe612f71457c1a5cc270a73e19893f710e0c5aea4606f30ec: Status 404 returned error can't find the container with id 488e01f9a329463fe612f71457c1a5cc270a73e19893f710e0c5aea4606f30ec Jul 08 09:41:51.428406 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.428375 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:51.428467 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.428439 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:51.428510 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.428468 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:51.612604 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.612511 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" event={"ID":"76ee0d61-5c67-4de4-afd6-25724ca0a0a0","Type":"ContainerStarted","Data":"dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791"} Jul 08 09:41:51.612604 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.612562 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" event={"ID":"76ee0d61-5c67-4de4-afd6-25724ca0a0a0","Type":"ContainerStarted","Data":"488e01f9a329463fe612f71457c1a5cc270a73e19893f710e0c5aea4606f30ec"} Jul 08 09:41:51.653025 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:51.652963 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" podStartSLOduration=1.652943284 podStartE2EDuration="1.652943284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:41:51.649275657 +0000 UTC m=+2109.899234940" watchObservedRunningTime="2026-07-08 09:41:51.652943284 +0000 UTC m=+2109.902902567" Jul 08 09:41:52.271664 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:52.271625 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:52.281153 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:52.281118 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:52.617822 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:52.617723 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:52.618623 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:52.618606 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:41:59.652511 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:59.652413 2572 generic.go:356] "Generic (PLEG): container finished" podID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerID="9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255" exitCode=0 Jul 08 09:41:59.652511 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:59.652487 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" event={"ID":"232dc30c-7f20-403d-add2-fe8c6c0042f1","Type":"ContainerDied","Data":"9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255"} Jul 08 09:41:59.652994 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:59.652978 2572 scope.go:117] "RemoveContainer" containerID="9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255" Jul 08 09:41:59.654104 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:59.654076 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:59.654169 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:59.654142 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:41:59.654204 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:41:59.654173 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:42:00.578801 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:00.578741 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:42:00.665532 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:00.665445 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" event={"ID":"232dc30c-7f20-403d-add2-fe8c6c0042f1","Type":"ContainerStarted","Data":"1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f"} Jul 08 09:42:00.943733 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:00.943652 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:42:00.944702 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:00.944675 2572 patch_prober.go:28] interesting pod/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 08 09:42:00.944868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:00.944732 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 08 09:42:01.943173 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:01.943134 2572 patch_prober.go:28] interesting pod/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 08 09:42:01.943666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:01.943193 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 08 09:42:02.943456 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:02.943418 2572 patch_prober.go:28] interesting pod/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 08 09:42:02.943863 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:02.943503 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 08 09:42:03.943591 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:03.943546 2572 patch_prober.go:28] interesting pod/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 08 09:42:03.944016 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:03.943621 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 08 09:42:04.947698 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:04.947615 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:42:04.948156 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:04.948010 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:42:04.948942 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:04.948920 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:42:05.697401 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:05.697367 2572 generic.go:356] "Generic (PLEG): container finished" podID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerID="dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791" exitCode=0 Jul 08 09:42:05.697585 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:05.697440 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" event={"ID":"76ee0d61-5c67-4de4-afd6-25724ca0a0a0","Type":"ContainerDied","Data":"dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791"} Jul 08 09:42:05.697992 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:05.697977 2572 scope.go:117] "RemoveContainer" containerID="dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791" Jul 08 09:42:05.698928 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:05.698901 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:42:05.699001 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:05.698983 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:42:05.699042 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:05.699031 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:42:06.705615 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:06.705581 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" event={"ID":"76ee0d61-5c67-4de4-afd6-25724ca0a0a0","Type":"ContainerStarted","Data":"50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84"} Jul 08 09:42:07.271319 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:07.271285 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:42:07.272181 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:07.272157 2572 patch_prober.go:28] interesting pod/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 09:42:07.272299 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:07.272218 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 09:42:07.618591 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:07.618483 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:42:08.271714 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:08.271673 2572 patch_prober.go:28] interesting pod/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 09:42:08.272125 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:08.271744 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 09:42:09.272005 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:09.271969 2572 patch_prober.go:28] interesting pod/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 09:42:09.272404 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:09.272038 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 09:42:10.271699 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:10.271663 2572 patch_prober.go:28] interesting pod/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 09:42:10.271901 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:10.271730 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 09:42:11.275137 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:11.275109 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:42:11.276082 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:42:11.276062 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:45:43.106138 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:45:43.106065 2572 scope.go:117] "RemoveContainer" containerID="fd0e9e53b405d7a4c122ba1de9eafd4e4b14a6c094414eacf5d73f4851093776" Jul 08 09:48:33.064363 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:33.064326 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f"] Jul 08 09:48:33.064864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:33.064536 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" containerID="cri-o://1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f" gracePeriod=30 Jul 08 09:48:34.950917 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:34.950826 2572 patch_prober.go:28] interesting pod/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 09:48:34.950917 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:34.950888 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 09:48:38.308202 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.308168 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:48:38.478919 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.478817 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/232dc30c-7f20-403d-add2-fe8c6c0042f1-istiod-ca-cert\") pod \"232dc30c-7f20-403d-add2-fe8c6c0042f1\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " Jul 08 09:48:38.478919 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.478881 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-envoy\") pod \"232dc30c-7f20-403d-add2-fe8c6c0042f1\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " Jul 08 09:48:38.479159 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.478939 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-podinfo\") pod \"232dc30c-7f20-403d-add2-fe8c6c0042f1\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " Jul 08 09:48:38.479159 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.478976 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-token\") pod \"232dc30c-7f20-403d-add2-fe8c6c0042f1\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " Jul 08 09:48:38.479159 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.478998 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-data\") pod \"232dc30c-7f20-403d-add2-fe8c6c0042f1\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " Jul 08 09:48:38.479159 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.479038 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw5g9\" (UniqueName: \"kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-kube-api-access-vw5g9\") pod \"232dc30c-7f20-403d-add2-fe8c6c0042f1\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " Jul 08 09:48:38.479159 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.479066 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-socket\") pod \"232dc30c-7f20-403d-add2-fe8c6c0042f1\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " Jul 08 09:48:38.479159 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.479103 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-certs\") pod \"232dc30c-7f20-403d-add2-fe8c6c0042f1\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " Jul 08 09:48:38.479159 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.479131 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-credential-socket\") pod \"232dc30c-7f20-403d-add2-fe8c6c0042f1\" (UID: \"232dc30c-7f20-403d-add2-fe8c6c0042f1\") " Jul 08 09:48:38.479508 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.479295 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/232dc30c-7f20-403d-add2-fe8c6c0042f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "232dc30c-7f20-403d-add2-fe8c6c0042f1" (UID: "232dc30c-7f20-403d-add2-fe8c6c0042f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:48:38.479508 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.479394 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "232dc30c-7f20-403d-add2-fe8c6c0042f1" (UID: "232dc30c-7f20-403d-add2-fe8c6c0042f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:48:38.479612 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.479592 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "232dc30c-7f20-403d-add2-fe8c6c0042f1" (UID: "232dc30c-7f20-403d-add2-fe8c6c0042f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:48:38.479664 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.479604 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "232dc30c-7f20-403d-add2-fe8c6c0042f1" (UID: "232dc30c-7f20-403d-add2-fe8c6c0042f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:48:38.480145 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.480114 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "232dc30c-7f20-403d-add2-fe8c6c0042f1" (UID: "232dc30c-7f20-403d-add2-fe8c6c0042f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:48:38.481234 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.481189 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "232dc30c-7f20-403d-add2-fe8c6c0042f1" (UID: "232dc30c-7f20-403d-add2-fe8c6c0042f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:48:38.481360 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.481337 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-kube-api-access-vw5g9" (OuterVolumeSpecName: "kube-api-access-vw5g9") pod "232dc30c-7f20-403d-add2-fe8c6c0042f1" (UID: "232dc30c-7f20-403d-add2-fe8c6c0042f1"). InnerVolumeSpecName "kube-api-access-vw5g9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:48:38.481634 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.481618 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "232dc30c-7f20-403d-add2-fe8c6c0042f1" (UID: "232dc30c-7f20-403d-add2-fe8c6c0042f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:48:38.481687 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.481634 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "232dc30c-7f20-403d-add2-fe8c6c0042f1" (UID: "232dc30c-7f20-403d-add2-fe8c6c0042f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:48:38.573157 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.573122 2572 generic.go:356] "Generic (PLEG): container finished" podID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerID="1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f" exitCode=0 Jul 08 09:48:38.573361 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.573189 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" Jul 08 09:48:38.573361 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.573204 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" event={"ID":"232dc30c-7f20-403d-add2-fe8c6c0042f1","Type":"ContainerDied","Data":"1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f"} Jul 08 09:48:38.573361 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.573279 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f" event={"ID":"232dc30c-7f20-403d-add2-fe8c6c0042f1","Type":"ContainerDied","Data":"38334e94087a8d1b44528153b0791eccfc5ea62a1adc7390f12a67449dda75ee"} Jul 08 09:48:38.573361 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.573309 2572 scope.go:117] "RemoveContainer" containerID="1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f" Jul 08 09:48:38.580536 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.580497 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-credential-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:38.580536 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.580522 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/232dc30c-7f20-403d-add2-fe8c6c0042f1-istiod-ca-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:38.580536 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.580538 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-envoy\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:38.580923 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.580553 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-podinfo\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:38.580923 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.580563 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-token\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:38.580923 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.580574 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-istio-data\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:38.580923 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.580585 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw5g9\" (UniqueName: \"kubernetes.io/projected/232dc30c-7f20-403d-add2-fe8c6c0042f1-kube-api-access-vw5g9\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:38.580923 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.580597 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:38.580923 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.580608 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/232dc30c-7f20-403d-add2-fe8c6c0042f1-workload-certs\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:38.582581 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.582567 2572 scope.go:117] "RemoveContainer" containerID="9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255" Jul 08 09:48:38.590333 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.590318 2572 scope.go:117] "RemoveContainer" containerID="1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f" Jul 08 09:48:38.590622 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:48:38.590604 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f\": container with ID starting with 1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f not found: ID does not exist" containerID="1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f" Jul 08 09:48:38.590669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.590632 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f"} err="failed to get container status \"1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f\": rpc error: code = NotFound desc = could not find container \"1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f\": container with ID starting with 1aaaba94bcba2a0b6e0ed2dde022b8458293871c11212bbca7263843f702654f not found: ID does not exist" Jul 08 09:48:38.590669 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.590650 2572 scope.go:117] "RemoveContainer" containerID="9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255" Jul 08 09:48:38.590885 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:48:38.590869 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255\": container with ID starting with 9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255 not found: ID does not exist" containerID="9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255" Jul 08 09:48:38.590933 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.590888 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255"} err="failed to get container status \"9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255\": rpc error: code = NotFound desc = could not find container \"9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255\": container with ID starting with 9528309de9a918088052bbb4363b92b7135a42db7115d13aa1553e0f29d0b255 not found: ID does not exist" Jul 08 09:48:38.595883 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.595860 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f"] Jul 08 09:48:38.599929 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:38.599910 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8c4427-openshift-default-5bd7bf8bc5-hcq5f"] Jul 08 09:48:40.309417 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:40.309386 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" path="/var/lib/kubelet/pods/232dc30c-7f20-403d-add2-fe8c6c0042f1/volumes" Jul 08 09:48:40.872304 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:40.872264 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5"] Jul 08 09:48:40.872521 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:40.872495 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" containerID="cri-o://50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84" gracePeriod=30 Jul 08 09:48:41.278383 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:41.278346 2572 patch_prober.go:28] interesting pod/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 09:48:41.278566 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:41.278427 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 09:48:46.120245 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.120220 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:48:46.133175 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133150 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hznvj\" (UniqueName: \"kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-kube-api-access-hznvj\") pod \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " Jul 08 09:48:46.133293 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133187 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-token\") pod \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " Jul 08 09:48:46.133293 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133235 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-podinfo\") pod \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " Jul 08 09:48:46.133293 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133254 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-socket\") pod \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " Jul 08 09:48:46.133293 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133283 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-data\") pod \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " Jul 08 09:48:46.133550 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133302 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-envoy\") pod \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " Jul 08 09:48:46.133550 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133319 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-credential-socket\") pod \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " Jul 08 09:48:46.133550 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133362 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istiod-ca-cert\") pod \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " Jul 08 09:48:46.133550 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133393 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-certs\") pod \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\" (UID: \"76ee0d61-5c67-4de4-afd6-25724ca0a0a0\") " Jul 08 09:48:46.133772 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133606 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76ee0d61-5c67-4de4-afd6-25724ca0a0a0" (UID: "76ee0d61-5c67-4de4-afd6-25724ca0a0a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:48:46.133969 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133853 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76ee0d61-5c67-4de4-afd6-25724ca0a0a0" (UID: "76ee0d61-5c67-4de4-afd6-25724ca0a0a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:48:46.133969 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133873 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76ee0d61-5c67-4de4-afd6-25724ca0a0a0" (UID: "76ee0d61-5c67-4de4-afd6-25724ca0a0a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:48:46.133969 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.133910 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76ee0d61-5c67-4de4-afd6-25724ca0a0a0" (UID: "76ee0d61-5c67-4de4-afd6-25724ca0a0a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:48:46.134400 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.134379 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "76ee0d61-5c67-4de4-afd6-25724ca0a0a0" (UID: "76ee0d61-5c67-4de4-afd6-25724ca0a0a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:48:46.135463 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.135409 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76ee0d61-5c67-4de4-afd6-25724ca0a0a0" (UID: "76ee0d61-5c67-4de4-afd6-25724ca0a0a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:48:46.135463 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.135418 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "76ee0d61-5c67-4de4-afd6-25724ca0a0a0" (UID: "76ee0d61-5c67-4de4-afd6-25724ca0a0a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:48:46.135721 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.135696 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-kube-api-access-hznvj" (OuterVolumeSpecName: "kube-api-access-hznvj") pod "76ee0d61-5c67-4de4-afd6-25724ca0a0a0" (UID: "76ee0d61-5c67-4de4-afd6-25724ca0a0a0"). InnerVolumeSpecName "kube-api-access-hznvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:48:46.135721 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.135701 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76ee0d61-5c67-4de4-afd6-25724ca0a0a0" (UID: "76ee0d61-5c67-4de4-afd6-25724ca0a0a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:48:46.234269 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.234226 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-certs\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:46.234269 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.234261 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hznvj\" (UniqueName: \"kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-kube-api-access-hznvj\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:46.234269 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.234271 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-token\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:46.234269 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.234280 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-podinfo\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:46.234549 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.234289 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-workload-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:46.234549 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.234299 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-data\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:46.234549 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.234309 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istio-envoy\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:46.234549 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.234318 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-credential-socket\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:46.234549 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.234326 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76ee0d61-5c67-4de4-afd6-25724ca0a0a0-istiod-ca-cert\") on node \"ip-10-0-136-68.ec2.internal\" DevicePath \"\"" Jul 08 09:48:46.613025 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.612995 2572 generic.go:356] "Generic (PLEG): container finished" podID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerID="50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84" exitCode=0 Jul 08 09:48:46.613208 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.613043 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" Jul 08 09:48:46.613208 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.613062 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" event={"ID":"76ee0d61-5c67-4de4-afd6-25724ca0a0a0","Type":"ContainerDied","Data":"50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84"} Jul 08 09:48:46.613208 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.613106 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5" event={"ID":"76ee0d61-5c67-4de4-afd6-25724ca0a0a0","Type":"ContainerDied","Data":"488e01f9a329463fe612f71457c1a5cc270a73e19893f710e0c5aea4606f30ec"} Jul 08 09:48:46.613208 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.613130 2572 scope.go:117] "RemoveContainer" containerID="50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84" Jul 08 09:48:46.621533 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.621516 2572 scope.go:117] "RemoveContainer" containerID="dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791" Jul 08 09:48:46.629115 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.629100 2572 scope.go:117] "RemoveContainer" containerID="50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84" Jul 08 09:48:46.629394 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:48:46.629374 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84\": container with ID starting with 50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84 not found: ID does not exist" containerID="50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84" Jul 08 09:48:46.629450 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.629402 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84"} err="failed to get container status \"50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84\": rpc error: code = NotFound desc = could not find container \"50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84\": container with ID starting with 50b9bc892a0db00f06796fc08539efba65e33a3ba1136ae6d36105c2fdbf4d84 not found: ID does not exist" Jul 08 09:48:46.629450 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.629418 2572 scope.go:117] "RemoveContainer" containerID="dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791" Jul 08 09:48:46.629669 ip-10-0-136-68 kubenswrapper[2572]: E0708 09:48:46.629653 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791\": container with ID starting with dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791 not found: ID does not exist" containerID="dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791" Jul 08 09:48:46.629723 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.629673 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791"} err="failed to get container status \"dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791\": rpc error: code = NotFound desc = could not find container \"dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791\": container with ID starting with dcaeb90dc90d71e3f23e938f726615b67cad3072afe7743bb09e2814f48fc791 not found: ID does not exist" Jul 08 09:48:46.635914 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.635893 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5"] Jul 08 09:48:46.640443 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.640423 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f0ff0b-openshift-default-c7bf6fdbc-8gnm5"] Jul 08 09:48:46.761523 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:46.761493 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-68855b66b7-krmrd_e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7/maas-api/0.log" Jul 08 09:48:47.552829 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:47.552803 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d6485f98-f7q25_2e8c1ce1-b47a-4a04-a128-d7f172173cfd/copy-manifests/0.log" Jul 08 09:48:47.566703 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:47.566675 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d6485f98-f7q25_2e8c1ce1-b47a-4a04-a128-d7f172173cfd/manager/0.log" Jul 08 09:48:47.963942 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:47.963836 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-wjmwm_f472b240-b1e0-4476-ab6a-646262f03444/postgres/0.log" Jul 08 09:48:48.307305 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:48.307274 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" path="/var/lib/kubelet/pods/76ee0d61-5c67-4de4-afd6-25724ca0a0a0/volumes" Jul 08 09:48:48.814200 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:48.814173 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64_973fd9b5-2707-43ec-81b7-3db96c8385a0/extract/0.log" Jul 08 09:48:48.822282 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:48.822260 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64_973fd9b5-2707-43ec-81b7-3db96c8385a0/util/0.log" Jul 08 09:48:48.833385 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:48.833366 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64_973fd9b5-2707-43ec-81b7-3db96c8385a0/pull/0.log" Jul 08 09:48:48.953653 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:48.953622 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch_7ceef47b-7a79-4a93-b828-ef7082f35650/util/0.log" Jul 08 09:48:48.963166 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:48.963141 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch_7ceef47b-7a79-4a93-b828-ef7082f35650/pull/0.log" Jul 08 09:48:48.970839 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:48.970817 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch_7ceef47b-7a79-4a93-b828-ef7082f35650/extract/0.log" Jul 08 09:48:49.093696 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:49.093610 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88_97af5b9f-a944-4d3f-9547-9c9404eaeb3d/extract/0.log" Jul 08 09:48:49.105154 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:49.105135 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88_97af5b9f-a944-4d3f-9547-9c9404eaeb3d/util/0.log" Jul 08 09:48:49.113600 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:49.113583 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88_97af5b9f-a944-4d3f-9547-9c9404eaeb3d/pull/0.log" Jul 08 09:48:49.236615 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:49.236583 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg_5cb68d01-0bec-48b1-91ab-145355aad160/util/0.log" Jul 08 09:48:49.243924 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:49.243905 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg_5cb68d01-0bec-48b1-91ab-145355aad160/pull/0.log" Jul 08 09:48:49.254995 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:49.254977 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg_5cb68d01-0bec-48b1-91ab-145355aad160/extract/0.log" Jul 08 09:48:50.801205 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:50.801160 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-c6h48_157ed47a-6e4e-4c64-8708-e119ddfb0575/discovery/0.log" Jul 08 09:48:51.306110 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:51.306077 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8r4k2_bed593dc-8139-4dbc-905d-e054ddbc591a/payload-pre-processing/0.log" Jul 08 09:48:51.564807 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:51.564709 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-db9df69b9-6wcdw_57259205-640d-4f5f-9e76-a607791fbe62/router/0.log" Jul 08 09:48:52.085967 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:52.085939 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qmxfk_b77d4eeb-046b-4857-b4dc-3d57112eab56/main/0.log" Jul 08 09:48:52.330686 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:48:52.330657 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbw7782_9b21d8ba-7db2-40d9-a964-204f0a3ac8b4/main/0.log" Jul 08 09:49:00.638156 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:00.638126 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dwhwz_c8c7dd60-3fc1-45b5-baae-d2867dd09577/global-pull-secret-syncer/0.log" Jul 08 09:49:00.898120 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:00.898044 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pbfjd_ef4d7d33-c0ca-4739-9c59-4b1c1b20a142/konnectivity-agent/0.log" Jul 08 09:49:00.952374 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:00.952349 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-68.ec2.internal_802e55c5ab26814202be9eac69956172/haproxy/0.log" Jul 08 09:49:05.294868 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.294839 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64_973fd9b5-2707-43ec-81b7-3db96c8385a0/extract/0.log" Jul 08 09:49:05.326902 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.326828 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64_973fd9b5-2707-43ec-81b7-3db96c8385a0/util/0.log" Jul 08 09:49:05.350608 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.350577 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599nb64_973fd9b5-2707-43ec-81b7-3db96c8385a0/pull/0.log" Jul 08 09:49:05.383846 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.383818 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch_7ceef47b-7a79-4a93-b828-ef7082f35650/extract/0.log" Jul 08 09:49:05.410457 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.410431 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch_7ceef47b-7a79-4a93-b828-ef7082f35650/util/0.log" Jul 08 09:49:05.436789 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.436762 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rxwch_7ceef47b-7a79-4a93-b828-ef7082f35650/pull/0.log" Jul 08 09:49:05.465482 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.465460 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88_97af5b9f-a944-4d3f-9547-9c9404eaeb3d/extract/0.log" Jul 08 09:49:05.491967 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.491934 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88_97af5b9f-a944-4d3f-9547-9c9404eaeb3d/util/0.log" Jul 08 09:49:05.518632 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.518601 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwh88_97af5b9f-a944-4d3f-9547-9c9404eaeb3d/pull/0.log" Jul 08 09:49:05.552830 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.552802 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg_5cb68d01-0bec-48b1-91ab-145355aad160/extract/0.log" Jul 08 09:49:05.582770 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.582674 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg_5cb68d01-0bec-48b1-91ab-145355aad160/util/0.log" Jul 08 09:49:05.615399 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:05.615372 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cntxg_5cb68d01-0bec-48b1-91ab-145355aad160/pull/0.log" Jul 08 09:49:08.781319 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:08.781289 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6zkz_6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd/node-exporter/0.log" Jul 08 09:49:08.809889 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:08.809867 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6zkz_6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd/kube-rbac-proxy/0.log" Jul 08 09:49:08.834732 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:08.834706 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6zkz_6a8ad7f6-9add-4f66-a2b5-179a3cfdcadd/init-textfile/0.log" Jul 08 09:49:09.419258 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:09.419227 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6868869c78-5bsvt_78f7a695-a225-4296-8e26-3cce33373aa8/telemeter-client/0.log" Jul 08 09:49:09.443936 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:09.443906 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6868869c78-5bsvt_78f7a695-a225-4296-8e26-3cce33373aa8/reload/0.log" Jul 08 09:49:09.473711 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:09.473683 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6868869c78-5bsvt_78f7a695-a225-4296-8e26-3cce33373aa8/kube-rbac-proxy/0.log" Jul 08 09:49:11.285728 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.285691 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq"] Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.285991 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286003 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286013 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286020 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286029 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286035 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286042 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286046 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286096 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286102 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="76ee0d61-5c67-4de4-afd6-25724ca0a0a0" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286111 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="232dc30c-7f20-403d-add2-fe8c6c0042f1" containerName="istio-proxy" Jul 08 09:49:11.286127 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.286116 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3961b8e8-b909-43cf-bb43-5a36c0eb563b" containerName="istio-proxy" Jul 08 09:49:11.288279 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.288262 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.291935 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.291914 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prsl4\"/\"kube-root-ca.crt\"" Jul 08 09:49:11.292998 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.292980 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-prsl4\"/\"default-dockercfg-6smtf\"" Jul 08 09:49:11.293072 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.293005 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prsl4\"/\"openshift-service-ca.crt\"" Jul 08 09:49:11.304253 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.304227 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq"] Jul 08 09:49:11.336176 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.336141 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-lib-modules\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.336339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.336196 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-sys\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.336339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.336266 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-proc\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.336339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.336291 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-podres\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.336339 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.336310 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqjsg\" (UniqueName: \"kubernetes.io/projected/68de07cb-d2dd-448b-a972-5905dc68e904-kube-api-access-nqjsg\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.437552 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.437512 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-proc\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.437552 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.437536 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-proc\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.437837 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.437587 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-podres\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.437837 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.437618 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqjsg\" (UniqueName: \"kubernetes.io/projected/68de07cb-d2dd-448b-a972-5905dc68e904-kube-api-access-nqjsg\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.437837 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.437651 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-lib-modules\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.437837 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.437733 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-sys\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.437837 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.437746 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-podres\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.438029 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.437860 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-sys\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.438029 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.437859 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68de07cb-d2dd-448b-a972-5905dc68e904-lib-modules\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.448864 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.448841 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqjsg\" (UniqueName: \"kubernetes.io/projected/68de07cb-d2dd-448b-a972-5905dc68e904-kube-api-access-nqjsg\") pod \"perf-node-gather-daemonset-skgbq\" (UID: \"68de07cb-d2dd-448b-a972-5905dc68e904\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.597672 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.597573 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:11.724406 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:11.724370 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq"] Jul 08 09:49:12.737272 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:12.737232 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" event={"ID":"68de07cb-d2dd-448b-a972-5905dc68e904","Type":"ContainerStarted","Data":"8e27ba7ff7c4013fc662fa3e72eb8a78db2aaa975824ba8d1569ae895da10f9a"} Jul 08 09:49:12.737666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:12.737291 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" event={"ID":"68de07cb-d2dd-448b-a972-5905dc68e904","Type":"ContainerStarted","Data":"5764f267984a6b6809af62128d9cd85c443258e31ece971f597427d5a6ee0f2e"} Jul 08 09:49:12.737666 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:12.737346 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:12.760409 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:12.760355 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" podStartSLOduration=1.76033974 podStartE2EDuration="1.76033974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:49:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:49:12.759086465 +0000 UTC m=+2551.009045751" watchObservedRunningTime="2026-07-08 09:49:12.76033974 +0000 UTC m=+2551.010299022" Jul 08 09:49:13.946066 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:13.946040 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zg4rz_be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8/dns/0.log" Jul 08 09:49:13.974215 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:13.974184 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zg4rz_be2b4a12-517d-4fe5-9ca3-24e1fa3bb7d8/kube-rbac-proxy/0.log" Jul 08 09:49:14.063102 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:14.063077 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ht7f6_048d9fac-8d08-46a7-a4c6-bae6f85821b6/dns-node-resolver/0.log" Jul 08 09:49:14.690522 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:14.690498 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-65jmn_61ca3fd1-d587-4a48-a64e-d2eaf482dc92/node-ca/0.log" Jul 08 09:49:15.891047 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:15.891019 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-c6h48_157ed47a-6e4e-4c64-8708-e119ddfb0575/discovery/0.log" Jul 08 09:49:16.067011 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:16.066979 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8r4k2_bed593dc-8139-4dbc-905d-e054ddbc591a/payload-pre-processing/0.log" Jul 08 09:49:16.165434 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:16.165361 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-db9df69b9-6wcdw_57259205-640d-4f5f-9e76-a607791fbe62/router/0.log" Jul 08 09:49:16.917579 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:16.917548 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8g5pt_152b5ffb-392f-4f27-9379-ca71ab6070b6/serve-healthcheck-canary/0.log" Jul 08 09:49:17.562180 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:17.562152 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9ssv9_114b9a1c-0321-4b1d-a570-97c10de99789/kube-rbac-proxy/0.log" Jul 08 09:49:17.589845 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:17.589820 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9ssv9_114b9a1c-0321-4b1d-a570-97c10de99789/exporter/0.log" Jul 08 09:49:17.617287 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:17.617260 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9ssv9_114b9a1c-0321-4b1d-a570-97c10de99789/extractor/0.log" Jul 08 09:49:18.751670 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:18.751641 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-skgbq" Jul 08 09:49:20.429323 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:20.429297 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-68855b66b7-krmrd_e6bc91fa-fb62-4171-8e85-6a7fdbdc53a7/maas-api/0.log" Jul 08 09:49:20.901971 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:20.901936 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d6485f98-f7q25_2e8c1ce1-b47a-4a04-a128-d7f172173cfd/manager/0.log" Jul 08 09:49:20.925619 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:20.925594 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d6485f98-f7q25_2e8c1ce1-b47a-4a04-a128-d7f172173cfd/copy-manifests/0.log" Jul 08 09:49:21.110538 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:21.110514 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-wjmwm_f472b240-b1e0-4476-ab6a-646262f03444/postgres/0.log" Jul 08 09:49:21.867561 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:21.867528 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-c6h48_157ed47a-6e4e-4c64-8708-e119ddfb0575/discovery/0.log" Jul 08 09:49:22.016186 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:22.016158 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8r4k2_bed593dc-8139-4dbc-905d-e054ddbc591a/payload-pre-processing/0.log" Jul 08 09:49:22.110144 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:22.110117 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-db9df69b9-6wcdw_57259205-640d-4f5f-9e76-a607791fbe62/router/0.log" Jul 08 09:49:22.734084 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:22.734052 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d9788fbb6-djv77_f90c3f88-744e-43f7-9727-a80bffe90bc3/manager/0.log" Jul 08 09:49:23.530677 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:23.530652 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-c6h48_157ed47a-6e4e-4c64-8708-e119ddfb0575/discovery/0.log" Jul 08 09:49:23.659853 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:23.659824 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8r4k2_bed593dc-8139-4dbc-905d-e054ddbc591a/payload-pre-processing/0.log" Jul 08 09:49:23.749962 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:23.749936 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-db9df69b9-6wcdw_57259205-640d-4f5f-9e76-a607791fbe62/router/0.log" Jul 08 09:49:29.713929 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:29.713902 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9vq6j_3b97acfe-ed2a-415d-9223-746ddcdb5115/kube-multus/0.log" Jul 08 09:49:29.914873 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:29.914847 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpzs_a6c58c8c-52bc-4924-92f2-13063e6bc4de/kube-multus-additional-cni-plugins/0.log" Jul 08 09:49:29.941258 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:29.941236 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpzs_a6c58c8c-52bc-4924-92f2-13063e6bc4de/egress-router-binary-copy/0.log" Jul 08 09:49:29.967135 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:29.967079 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpzs_a6c58c8c-52bc-4924-92f2-13063e6bc4de/cni-plugins/0.log" Jul 08 09:49:29.994340 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:29.994316 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpzs_a6c58c8c-52bc-4924-92f2-13063e6bc4de/bond-cni-plugin/0.log" Jul 08 09:49:30.017678 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:30.017651 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpzs_a6c58c8c-52bc-4924-92f2-13063e6bc4de/routeoverride-cni/0.log" Jul 08 09:49:30.046731 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:30.046706 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpzs_a6c58c8c-52bc-4924-92f2-13063e6bc4de/whereabouts-cni-bincopy/0.log" Jul 08 09:49:30.073050 ip-10-0-136-68 kubenswrapper[2572]: I0708 09:49:30.073025 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpzs_a6c58c8c-52bc-4924-92f2-13063e6bc4de/whereabouts-cni/0.log"